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ABSTRACT 



This report provides an evaluation of the Online at PA 
Libraries project which was initiated in the Fall of 1996 by the Pennsylvania 
Department of Education's Office of Commonwealth Libraries and the Schuylkill 
Intermediate Unit. The report documents that the following goals were 
accomplished: (1) provide public access to the Internet through 

Pennsylvania's public, libraries; ( 2 ) p rev -i de br a r y u r // i til t i i ae 1 y 

information for educational research, economic development, and recreational 
purposes; and (3) ensure that trained library staff are available to help the 
public find the information they need in an online environment. Key 
assessment questions were answered by the use of a range of data collection 
techniques including surveys of librarians and users, site visits, focus 
groups, interviews, and other techniques. From the viewpoint of project 
participants, the project was clearly successful. The list of various 
benefits and impacts that have come to the library, the users, and the 
community overall is significant and consistent across most libraries. The 
study concludes with a discussion of key issues raised by the Online at PA 
Libraries project and lessons that have been learned from this effort that 
can be incorporated into future efforts. Specific recommendations include 
building on the project with future projects, developing new roles for 
Pennsylvania libraries and librarians, and developing a statewide strategic 
planning effort to continue to take advantage of the evolving global 
networked environment for the benefit of Pennsylvania residents. The user 
questionnaire is appended. (Author/AEF) 
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EXECUTIVE SUMMARY 



This report provides an evaluation of the OnLine at PA Libraries project which 
was initiated in the Fall of 1996. The project, funded by Bell Atlantic, and 
administered by the Pennsylvania Department of Education’s Office of 
Commonwealth Libraries and the Schuylkill Intermediate Unit, has had a significant 
and important impact on the provision of Internet-based information services and 
resources to the residents of Pennsylvania. The original project goals were to: 

• Provide public access to the Internet through Pennsylvania’s public libraries. 

• Provide library users with timely information for educational, research, 
economic development, and recreational purposes. 

• Ensure that trained library staff are available to help the public find the 
information they need in an online environment. 

The report documents that these goals were, in fact, accomplished. Moreover, the 
report identifies many additional impacts and benefits that resulted from the project. 

The evaluation, which began October 1996 and was completed October 1997, was 
guided by specific research questions such as: What are the scope, reach, and 
dimensions to the OnLine at PA Libraries project? How much Internet activity and 
of what types are taking place in participating libraries? What types of users access 
the Internet connection and to what types of services and activities do they connect? 
What are the costs for such connectivity and various types of network activities and 
services? How has access to and use of networked information resources and 
services affected participating libraries in terms of economic competitiveness, 
education, productivity, quality of life, and other indicators of socio-economic well- 
being? 

These, and other key assessment questions were answered by use of a range of 
data collection techniques including surveys of librarians and users, site visits, focus 
groups, interviews, and other techniques. 

From the viewpoint of project participants — be they library directors, library 
staff, users, or local government officials — the project has clearly been successful. 
The list of the various benefits and impacts that have come to the library, the users, 
and the community overall, is significant and consistent across most libraries. Some 
of the most important indicators of the success of the project are that it: 

• Linked networked information to users and resolved a range of their 
information needs that otherwise would not have been resolved; there were 
numerous “success stories” of how public access to the Internet greatly 
assisted users; 



McClure and Bertot 



October 1997 



u 



O 

ERIC 



,3 



Evaluation of the OnLine at PA Libraries Project 



• Removed some of the isolation of the community by providing a “link” between 
the participating libraries’ communities and the ideas, resources, and 
information in a global setting; 

• Provided access to a range of networked information resources and services 
that otherwise could never have been obtained by participating hbraries; 

• Brought new users into participating libraries that otherwise may not have 
used “traditional” library services or resom-ces; 

• Increased the visibility and credibility of participating libraries; 

• Provided valuable publicity for participating libraries as a public access point 
to state-of-the-art technology and information services; 

• Served as a catalyst for participating libraries and their staff to move into the 
electronic networked environment; and 

• Provided a workstation and additional information technology that otherwise 
could not have been prurchased or obtained by participating libraries. 

Other key indicators of the success of the project include the following: 

• 26% of users of the public access workstation were new users, i.e., they had not 
previously visited the library. 

• The public access workstation attracts both expert — 44% — and beginning 
users — 48%. 

• 35% of users of the public access workstations had no other means to access 
the Internet except from participating OnLine at PA Libraries. Perhaps 
equally interesting is that the other 65% do have access to the Internet 
elsewhere (e.g., work, home, school), and still choose to use the public access 
workstation. 

• Users of the workstations (on a scale of 1= Very much so and 5= Not at all) 
rated finding the information they needed on the Internet, the ease of using the 
Internet via the library, and the importance of having access via the library 
between 1.7 and 2.2. Overall, therefore, users found that gaining access to the 
Internet, using the workstation, and the information found on the Internet to 
be of value. 

• In terms of major benefits from the project, 48% of participating library 
directors thought the expanded availability to reference sources was critical; 
and 32% thought the major benefit was attracting new users to the library. 
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The study concludes with a discussion of key issues raised by the OnLine at PA 
Libraries project and lessons that have been learned from this effort that can be 
incorporated into future efforts. Specific recommendations include building on the 
OnLine at PA Libraries project with future projects, developing new roles for 
Pennsylvania librarians and libraries, and developing a statewide strategic planning 
effort to continue to take advantage of the evolving global networked environment for 
the benefit of Pennsylvania residents. 
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PREFACE 



In Fall 1995, the President and CEO of Bell Atlantic spoke to Pennsylvania's First 
Lady about his interest in helping libraries. The First Lady is a professional librarian 
who directed a Pennsylvania library system before her husband was elected 
Governor. The First Lady and the Administration saw the need for libraries to 
provide access to the Internet. They suggested that Bell Atlantic work with Gary D. 
Wolfe, Deputy Secretary of Education and Pennsylvania's Commissioner for 
Libraries. The Deputy Secretary met with Bell Atlantic’s President and Vice 
President for External Affairs to discuss a possible proposal. The Deputy Secretary 
said he was interested in bringing the Internet to small rural libraries. The President 
of Bell Atlantic was excited at the prospect. Such a project was what he had in mind. 

The Deputy Secretary and his staff from the Office of Commonwealth Libraries, 
also known as the State Library of Pennsylvania, began work on a draft proposal. 
They worked with a committee that included representatives from the Schuylkill 
Intermediate Unit (SIU); the Executive Director of the Health Sciences Library 
Consortium; the Education Secretary's Special Assistant for Technology; the Deputy 
Secretary and selected staff from the Commonwealth's Office for Information 
Technology; and representatives from the Governor's Office. 

The committee discussed and revised the draft proposal after a conference call 
with Bell Atlantic employees. The Pennsylvania Department of Education submitted 
the final proposal to Bell Atlantic in December 1995. After some negotiation with Bell 
Atlantic, the proposal was approved and granted the Commonwealth $750,000 to 
bring the Internet to Pennsylvanians through small, rural public libraries. 

On April 17, 1996, Bell Atlantic presented the $750,000 check to the First Lady at 
a ceremony at the East Shore Branch Library of the Dauphin County Library 
System. After the grant award, the First Lady demonstrated the World Wide Web to 
elementary students from a neighboring school. 

The Office of Commonwealth Libraries contracted with the SIU to implement the 
grant and provide technical consultation and assistance, to handle equipment orders, 
and to coordinate training with State Library staff. The Director of Library 
Development handled implementation of the project at the State Library. The SIU 
hired an individual to provide technical coordination of the project while the SIU 
manager continued to provide professional technical consultation. Most planning and 
implementation of project objectives involved these three people. 

Public libraries had to apply to participate in the project. Appendix A is a copy of 
the application form used by libraries to participate in the project. Applications were 
due by Jxme 15, 1996, and applicants had to agree to: 
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• Place the equipment (comprised of a computer workstation with software 
and a printer) in an accessible public service area, on furniture supplied by 
the participating library, and make the workstation available for the public 
to use when the library was open. 

• Install one phone line to the workstation. 

• Participate in the project at least two years after the grant year and pny 
certain start-up and continuing costs (the grant application had an 
estimation of these costs. Local costs included phone line installation, 
telephone bill for local line, Internet provider fee, software/hardware 
maintenance and training/technical support after the grant year). 

• Have at least one public service staff member attend all required training 
sessions. 

• Agree to participate in public awareness activities. 

One hundred eighty-eight libraries applied and all were accepted into the project. 
Participating libraries were notified in July 1996.* 

The SIU ordered the equipment in August, the vendor configured it in August and 
September 1996, and it was shipped to participating libraries in October and early 
November. Basic training was held in October 1996. Libraries began offering the 
service as early as November 1996 with the last libraries coming online in February 
1997. A committee of reference librarians, district library center (DLC) consultant 
librarians, SIU staff, and Office of Commonwealth Libraries staff met in February 
and March to plan the advanced training, which was held in April and May 1997 . 

The SIU contracted, in October 1996, with Charles R. McClure and John Carlo 
Bertot to conduct an evaluation of the project. They completed the evaluation in 
October 1997, resulting in this report. 



* Due to lower than expected telecommunications costs (almost all libraries found 
Internet Service Providers within their local calling areas), the State Library was 
able to add 55 more local libraries to the project in August 1997. Most of those new 
libraries begain providing service to the public by September 1997. Data reported in 
this study do not inlcude these 55 libraries. 
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CHAPTER 1 



EVALUATION OF THE ONLINE AT PA LIBRARIES PROJECT: 
PUBLIC ACCESS TO THE INTERNET THROUGH PUBLIC LIBRARIES 



The OnLine at PA Libraries project (henceforth referred to as the project) has 
provided an important impetus to rural libraries in the state of Pennsylvania to 
become part of the evolving global networked information society. The project began 
at a fortuitous time when there was both funding available from Bell Atlantic and 
significant interest from the public library community in connecting to the Internet 
and providing new services to their communities. The project was also timely as it 
demonstrated what could he done, statewide, with some new resources, initiative, 
interest, and committed and dedicated librarians. 

The original project goals were to: 

• Provide public access to the Internet through Pennsylvania’s public hbraries. 

• Provide library users with timely information for educational, research, 
economic development, and recreational purposes. 

• Ensure that trained library staff are available to help the public find the 
information they need in an onhne environment. 

Specific project objectives to be accomplished during the 18 month effort that 
began April 1996 included: 

• Establish additional points of Internet connectivity to serve those regions of 
the state currently lacking local access. 

• Utilize optimal commrmications pathways to ensure equitable and inexpensive 
online access throughout the state. 

• Equip 300 public libraries to offer a graphical, user friendly interface to the 
Internet. 

• Provide basic and advanced training attended by at least 500 library staff 
members who increase their skills to better help the public navigate in the 
onhne environment. 

By September 1997, at the end of the project: 

• At least 500,000 Pennsylvanians per week will access online information 
sources through the Internet at their public libraries. 
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• Based on survey results, 80% of public access users will report the infomaation 
they find helps them make wise decisions in their personal, work, and public 
lives. 

These were ambitious goals and objectives. The State Library, Bell Atlantic, the 
Schuylkill lU, participating public libraries and librarians, and others associated with 
the project committed significant effort, time, and resources to making it successful. 

The project operated in a context where Pennsylvania ciirrently ranks 36th of the 
50 states in per capita support for public libraries; continues to fall behind other 
states in its support and development of new services for public libraries; and as a 
result, has low circulation per capita and smaller staffs than other states 
{Philadelphia Inquirer , June 1-4, 1997). Yet despite these and similar statistics of 
limited support for public libraries in Pennsylvania, the OnLine at PA Libraries 
project has been successful. 

The project has provided a statewide demonstration of what can be done to quickly 
and significantly improve public library services statewide; to connect public libraries 
to the Internet and provide a range of new services and resources to their 
communities; and to “jumpstart” libraries, librarians, and citizens into the global 
networked society. This report describes these (and other successes) and identifies 
related key issues for government officials, the library community, and local 
commimities to discuss as public libraries continue to move into the global networked 
society. 



INTRODUCTION 

The idea of a "networked environment" encompasses a range of electronic 
networked activities and services. Minimally, the networked environment includes 
information and media services, products, hardware and software, and resources that 
are received by users via electronic networks. In this environment, many 
information services are provided by local, regional, national, and international 
networks. Locally developed information services (i.e., from a library, computing 
services, private sector firms, individuals, or others) may comprise the majority of 
the networked environment. 

There is no easy or straightforward method to operationalize a definition for a 
network. It may be helpful to think of a network as comprising these components: 

• Technical Infrastructure : the hardware, software, equipment, communication 
lines, and technical aspects of the network; 

• Information Content : the information resources available on the network; 
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• Information Services : the activities in which users can engage and the 

services that users may use to complete various tasks; 

• Support : the assistance and support services provided to help users better use 
the network; and 

• Management : the human resources, governance, planning, and fiscal aspects 
of the network. 

These five basic components suggest the multi-dimensional nature of electronic 
networks in public libraries and the extent to which the networks can be described, 
defined, and then evaluated. 

Many organizations in general, and public libraries in particular, have built 
significant networks and connected to the Internet as part of the evolving National 
Information Infrastructure (Nil). As of Spring 1997, 72.3% of public libraries are 
connected to the Internet, as compared to 20.9% in 1994 (Bertot and McClure, 1997). 
The overall public library level of Internet connectivity varies greatly, however, by 
the population public libraries serve, with 1997 data suggesting that public libraries 
in larger population areas having significantly higher (100% for libraries with 
population of legal service areas of greater than one million) rates of Internet 
connectivity than public libraries in smaller population areas (56.3% for libraries with 
population of legal service areas of less than 5,000). There is also a significant 
difference in public library connectivity between urban (86.9% for central city 
libraries) and rural (66.0% for non central city libraries) libraries. Such differences 
also are true for Pennsylvania. Thus, public library Internet connectivity has not 
been equal nor even throughout the Nation nor Pennsylvania. As a result of the 
Project, however, access has become significantly more equal and even across the 
state. 

There is a great deal of electronic networking activity occurring in the public 
library, statewide, and K-12 environments. These innovative and creative initiatives 
demonstrate what can happen when the private sector, federal, state, and local 
governments, community-based stakeholder groups, and private sector organizations 
collaborate to create new means of working together, doing more with less, and 
reducing the overlap of services. In doing so, network creators assume that such 
initiatives will provide better government services and promote greater citizen 
prosperity, productivity, and education at all levels through the effective and efficient 
use of advanced networking technologies (Office of the Vice President, 1993; National 
Information Infrastructure Advisory Committee, 1995). 
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PUBLIC LIBRARY TELECOMMUNICATIONS AND 
NETWORK POLICY CONTEXT 

The current policy environment in which public library and statewide networking 
activities occur is one of rapid change. The telecommunications legislative and 
regulatory frameworks, as well as federal library-related legislation, continue to 
evolve and create new opportrmities for public library-based networking initiatives. 
Perhaps more challenging than this fluid policy context is the speed with which new 
technologies continue to emerge and provide new challenges and needs for technology 
planning and implementation at both local library and statewide levels. 



The Library Services and Technology Act 

On September 30, 1996, the President signed into law the Library Services and 
Technology Act (P.L. 104-208) (LSTA). LSTA marked a change in the direction of 
federally-funded library initiatives over its predecessor the Library Services and 
Construction Act (LSCA) in several key ways: 

• LSTA consolidates portions of the Higher Education Act (HEA) that related to 
a variety of library-related aspects. In doing so, LSTA creates the Institute for 
Museum and Library Services (IMLS) as the federal agency responsible for the 
administration of LSTA; 

• LSTA applies to nearly all types of libraries, not just public libraries; 

• LSTA expands the scope of library activities beyond those traditional services 
of books and bmldings to electronic networking activities; and 

■ • LSTA requires states to evaluate and report on the impact of LSTA-funded 
initiatives. 

Taken together, these key components of LSTA create a new federal-state-library 
funding environment that emphasizes collaboration, performance, and technological 
innovation. 



The Government Performance and Review Act 

Espousing the virtues and needs of effective and efficient government requires 
that citizens and federal government managers alike benchmark government 
services against some performance measure(s) and/or indicator(s). In a step towards 
developing such performance measures, the Congress passed the Government 
Performance and Results Act (P.L. 103-62) (GPRA). The GPRA stresses the need to 
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improve federal program effectiveness and public accountability by promoting a new 
focus on results, service, and customer satisfaction. 

Specifically, the GPRA requires federal agencies to establish pro^am-based 
performance goals for agency program areas that are quantifiable, objective, and 
measurable. Agencies must also create performance indicators that can rneasure 
and/or assess the outputs, service levels, and outcomes of agency program activities. 

The GPRA, therefore, requires each federal agency to have a clear mission that 
describes the purpose and function of the agency, develop set of outcome-oriented 
objectives that serve to attain the agency mission, and develop a set of quantifiable 
performance indicators and measures that will assist program managers determine 
whether and the extent to which their programs achieve program objectives and 
support the agency mission. 

GPRA has a substantial impact on the IMLS and, subsequently, LSTA. LSTA 
was created as a performance- and results-based initiative. There is likely to be 
continued emphasis on states receiving federal money to be able to demonstrate and 
measure specific outcomes from federally funded projects. Given this environment, 
the evaluation of the OnLine at PA Libraries project provides a range of methods and 
data collection techniques that can be used on similar projects by the State Library. 



The Telecommunications Act of 1996 

The Telecommunications Act of 1996 (P.L. 104-104) (TCA) was the first 
significant legislative overhaul to the Communications Act of 1934. The TCA 
essentially updated a variety of key aspects of the telecommunications industry, 
creating a more market-driven industry that relied on competition to foster lower 
telecommunications rates throughout the nation (Mueller, 1997). 

The universal service provision of the TCA specifically directed the Federal 
Communications Commission (FCC) to create a discount structure for 
telecommunications services for schools, libraries, and rural health care institutions 
(P.L. 104-104, Section 254). Based on the broad sidelines established by the TCA, 
the FCC issued its final universal service rulemaking on May 7, 1997. In this ruling, 
the FCC created a (Federal Communications Commission, 1997, Section X): 

• $2.25 bilhon annual discount fund for schools and hbranes; and 



• Telecommunications discount structure ranging from 20%-90% for 
telecommunications services (defined as telecommunications conduits - e.g., 
leased-lines — , internal wiring, and Internet connectivity). The discount rate a 
school or library can receive depends on the percentage of students on school 
lunch programs and the location (urban/rural) of the school or library . 
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The universal service provisions of the TCA, and the FCC implementation of those 
provisions, are aimed specifically at increasing the level of connectivity of schools and 
libraries to the Internet. 



Putting the LSTA, TCA, and GPRA Together 

Collectively, LSTA, TCA, and GPRA create a policy and regulatory environment 
that emphasizes market forces, performance measurement, and connectivity with 
the intent of creating an economically competitive nation in an information society 
(Bertot and McClure, 1996a). 

More specifically, LSTA, TCA, and GPRA combine to: 

• Promote local and statewide electronic networking activities to ultimately 
create a nationwide network of networks to which all Americans have access, 

• Expand the concept of telephone-based universal service to the electronic 
networked environment by providing schools and libraries with discounted 
telecommunication services; and 

• Evaluate the economic and social impacts of networking initiatives and funding 
for such initiatives on local, state, and national communities. 

This new environment is one, therefore, in which schools, libraries, and state library 
agencies need to collaborate more than ever to create an electronic networked 
infrastructure and subsequently provide a variety of network-based services over 
that infrastructure. 



The Pennsylvania Context 

In June 1997, the Pennsylvania Public LFtility Commission (PLFC) adopted the 
FCC discount matrix, making Pennsylvania eligible for intra-state as well as 
interstate telecommunications discounts for schools and libraries under the 
Telecommimications Act of 1996. The Pennsylvania program will not go beyond the 
eligibilities and discounts in the federal program at this time. The Office of 
Commonwealth Libraries has responsibility for reviewing and approving library 
plans. 
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To date, the Office of Commonwealth Libraries has provided advice to libraries in 
the form of memos and workshops. The memos were sent to all libraries that were 
thought to be eligible and the workshops have been for the DLC administrators and 
consultants so that they could bring back the information to their local libraries. The 
Office of Commonwealth Libraries also provided information so that each library 
could calculate its discount. All public libraries in PA have the potential of receiving 
discount rates between 40% and 80%. 

The telecommunications environment and use of the Internet in Pennsylvania 
libraries is a rapidly changing situation. Those libraries that participated in the 
OnLine at PA Libraries project (as well as others) are likely to provide significant 
increased demands for Internet-based services and connections. There are, however, 
significant issues yet to be addressed in terms of equality of access, costs, availability 
of equipment, and resource sharing. 



THE EVALUATION APPROACH 

The OnLine at PA Libraries project provides an excellent opportunhy not only to 
study the impacts of providing Internet connectivity to local communities, it also can 
assist the development of valid and reliable network performance measures and 
evaluation techniques. In addition, answering a number of key questions can assist 
Pennsylvania officials to better plan for future Internet development. These 
questions include: 

• What are the scope, reach, and dimensions to the OnLine at PA Libraries 
project? 

• How much Internet activity and of what types are taking place in 
participating hbraries? 

• What types of users access the Internet Connection and to what types of 
services and activities do they connect? 

• What are the costs for such connectivity and various types of network 
activities and services? 

• How has access to and use of networked information resources and services 
affected participating libraries in terms of economic competitiveness, 
education, productivity, quality of life, and other indicators of socio-economic 
well-being? 

• What types of performance measures are appropriate to determine the impact 
of this initiative on participating libraries, local communities, and individuals; 
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and what t5rpes of performance measurement tools can assist policy makers 
and researchers measure those identified performance measiires? 

There is a growing interest in identifying and measuring the impacts of public library 
Internet connectivity. Answers to questions such as “does the network improve the 
productivity of users?” or “is networking worth the cost?” are impossible to answer 
without first defining and then calculating some basic measurement techniques. The 
OnLine at PA Libraries project provides an opportunity to extend our knowledge 
about how to answer these and related questions. 

Approaches for evaluating networked information services can be based on the 
following criteria: 

• Extensiveness : how much of the service has been provided, e.g., number of 
users logging-in per week on a bulletin board or the nrunber of participants of a 
particular listserv. 

• Efficiency : the use of resources in providing or accessing networked 

information services, e.g., cost per session in providing access to remote users 
of an on-line catalog, or average time required to successfully telnet to a 
remote database. 

• Effectiveness : how well the networked information service met the objectives 
of the provider or the user, e.g., success rate of identifying and accessing the 
information needed by the user. 

• Service quality : how well a service or activity is done, e.g., percentage of 
transactions in which users acquire the information they need. 

• Impact : how a service made a difference in some other activity or situation, 
e.g., the degree to which network users enhanced their ability to gain 
employment or pursue business. 

• Usefulness : the degree to which the services are useful or appropriate for 
individual users, e.g., percentage of services of interest to different types of 
user audiences. 

• Adoption : the extent to which institutions or users integrate and adopt 
electronic networked resources or services into organizational or individual 
activities, e.g., classroom instruction. 

Evaluations of public library provision of networked information services, while 
needing to consider extensiveness and efficiency criteria, should give more attention 
to effectiveness, service quality, usefulness, impact, and adoption measures. 
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Developing measures to assess impacts from the provision of networked services 
from public libraries has proven to be a challenging endeavor because of the lack of 
basic measures that describe and assess the networked environment. Methods for 
simply counting types of users and their network activities require the resolution of a 
range of issues and policies. Nonetheless, this study provides a beginning perspective 
on the development of such measures and evaluation techniques. 

The evaluation of the OnLine at PA Libraries project continues work by McClure 
and Bertot which includes the national survey and assessment of Public Libraries 
and the Internet in 1994 (McClure, Bertot, and Zweizig), in 1996 (Bertot, McClure, and 
Zweizig), and in 1997 (Bertot and McClure); Internet Cost and Cost Models for Public 
Libraries (McClure, Bertot, and Beachboard, 1995); and the evaluation of the 
statewide network in Maryland, Sailor Network Assessment Final Report (Bertot and 
McClure, 1996b). 

EVALUATION METHODOLOGY 

This evaluation project serves as an initial assessment of the Project that will 
provide the Office of Commonwealth Libraries with baseline data on the value, use, 
and impact of the project on various user communities throughout the state. 



Evaluation Goals and Objectives 

In particular, the evaluation project will assist the Office of Commonwealth 
Libraries, public librarians, and others by achieving the following goals: 

• Determining the degree to which original project goals and objectives have been 
accomplished. 

• Evaluating the value of the project to such communities as the K-12, public 
library, business, minority, senior, and state and local government 

• Recommending refinements and modifications to the project to more closely 
meet the needs of the target communities 

• Measuring the nature and extent of K-12, public library, business, minority, 
senior, and state and local government community use of the Project, with 
emphasis on the incorporation of Project-based resources into community 
activities 

• Creating measurement devices that the Office of Commonwealth Libraries 
can use in the future to maintain ongoing or periodic assessment of the project. 
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To achieve these goals, the consultants performed the following tasks: 

(1) Assessed the impact of the project on participating public libraries in terms of 

• Provision of basic reference services 

• Provision of document delivery services 

• Assistance to users for 

-Instruction 
-Technical assistance 

• Assessment of whether the role of libraries has changed in relation to local 
government 

• Assessment of the impact of the project on library staff, administration, 
and trustees regarding 

-Staff development and/or continuing education 

-Staff allocation 

-Policies (service and access). 

(2) Assessed the project’s benefits to users by 

• Developing and administering measurement tools that determined 

- User groups within the K-12, public library, business, minority, senior, 
and state and local government communities 

- The methods through which the user groups became aware of the 
project 

• Interviewing a cross-section of the project’s target communities to 
determine the level of involvement of such communities with public access 
to the Internet to 

- Specify systemic use of the public access Internet workstation 
including, but not limited to, reference interactions, point of use 
instruction, and integration into the workplace 

- Assess the impact of the Project beyond public libraries (e.g., use by 
state and local government in improving their ability to perform the 
duties of their jobs, use by businesses in improving their ability to 
perform the duties of their jobs, use by citizen groups (e.g., senior, 
minority) in meeting their non-mediated information needs or for 
personal enrichment) 
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(3) Developing measurement devices and methods for SIU and the Office of 
Commonwealth Libraries to implement at a later date for continued evaluation 
of the project and its impact of services provided by libraries in Pennsylvania. 

(4) Providing a training session to selected SIU and the Office of Commonwealth 
library personnel in the use and execution of project data collection 
instruments to facilitate the transition of data collection activities to SIU and 
Commonwealth libraries personnel after completion of the evaluation. 

These evaluation goals and objectives provided a framework that directed the 
organization of study activities. 



Time Line and Project Tasking 

The evaluation project began on October 1, 1996, and continued through October 
1, 1997. The evaluation comprised three main phases. Phase I was project planning, 
literature review, introduction to project activities, and development/pretesting of 
data collection instruments. Phase II included administration of a range of data 
collection efforts and analysis of the data resulting from those efforts. Phase III 
focused on assessment of findings/conclusions, development and dissemination of the 
final report, which included a presentation on the project at the Pennsylvania Library 
Association Conference, September 1997. In conjunction with SIU and the Office of 
Commonwealth Libraries, the consultants developed a more detailed project tasking 
timeline during the first phase of the research project. 



Data Collection 

The study drew upon a combined qualitative and quantitative approach, selecting 
data collection techniques that best met the goals and objectives of the study. 
Qualitative techniques were employed, for example, in the use of ongoing expert 
review of the assessment techniques, measures, policy analysis, and measurement 
tool development. The qualitative approaches also assisted the researchers to 
develop such quantitative measurement tools as user surveys, the training surveys, 
and the survey of library directors. Quantitative techniques contributed to the 
development of measures related to public access workstation use, and costs, as well 
as user involvement with the electronic services. 

The evaluation relied primarily on a number of different data collection techniques, 
including; 
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• Focus groups. The consultants conducted a focus group in February 1997 
with librarian participants to inform the development of other data collection 
instruments; additional focus groups were conducted as part of the site visits. 

• Help dlesk use logs. SIU staff tracked the use and duration of requests during 
selected sample periods. 

• Interviews. The consultants conducted on-site or phone interviews with 
library administrators, Project system designers, and various user community 
members. 

• Surveys. The consultants incorporated in the site visits as well as three other 
key efforts: 

Survey of participants attending the October 1996 and April 1997 training 
sessions sponsored by the SIU and the Office of Commonwealth Libraries 

- Survey of participating library directors as to their overall perceptions and 
assessment of the project to date (March 1997). 

- Survey of users of the workstations in participating libraries (May 1997). 

• Site visits. The consultants observed the use and operation of the public 
access work stations, interviewed staff and other participating librarians, and 
conducted a focus group of users at four locations throughout the state of 
Pennsylvania. 

The combination of these data collection strategies allowed the study team to use 
multiple techniques to investigate various project topics and obtain quality data. 
Specific steps in the method and data collection were based on proven strategies (e.g., 
Rossi and Freeman, 1993; Kruegar, 1994; Creswell, 1994). Throughout the process 
appropriate steps were taken to ensure the collection of reliable and valid data as per 
standard research practice (Brinberg and McGrath, 1985). 



Project Prodlucts andl Dissemination 

The consultants provided the SIU and the Office of Commonwealth Libraries 
administrators with multiple project products, including: 

• Preliminary project findings and updates in January 1997 and June 1997. 
These updates served to inform SIU and the Office of Commonwealth 
Libraries staff as to the progress, preliminary findings, and key issues of the 
evaluation project. 
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• Data collection tools and instruments for continued SIU and the Office of 
Commonwealth Libraries data collection activities (included in the final report). 

• A training session for selected Office of Commonwealth Libraries staff on the 
use of the data collection tools and instruments. 

• Databases (on disk) of the survey data reported in Chapter 2. 

• A final report detailing study findings, methodologies, and data collection tools, 
findings, and recommendations. 

Furthermore, the consultants pursued several dissemination strategies for project 
fin dings including presentation of preliminary study findings at the Pennsylvania 
Library Association annual conference; presentation of project findings at national 
and state conferences such as the Public Library Association National Conference in 
March 1998; and submission of articles encompassing project findings to research 
journals, professional publications, and the ERIC Clearinghouse. 



ORGANIZATION AND PURPOSE OF THE FINAL REPORT 

This first chapter serves as an introduction to the study and provides backgroimd 
information to set the study in context. Chapter 2 reports on specific data collection 
efforts including: 

• Survey of participants attending SIU and the Office of Commonwealth 
Libraries training sessions (conducted October 1996 and April 1997). 

• Survey of participating library directors (conducted March 1997). 

• Survey of workstation users (conducted May 1997). 

• Log of SIU help desk requests (October, 1996 - June, 1997). 

Chapter 2 also contains additional detail about the data collection methodologies and 
instruments. 

Chapter 3 reports on the site visits at four participating libraries conducted in 
June - July 1997. These site visits included a range of data collection activities which 
are described in greater detail in Chapter 3. Chapter 4 offers a brief summary of 
findings. It also identifies key issues warranting additional discussion, describes some 
lessons that have been learned from the project, and it provides recommendations as 
to next steps and strategies for building on the existing momentum created by the 
project. The appendices include background information regarding the project as well 
as copies of the various data collection instruments. 
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The purpose of this final report is to provide a concise description of the evaluation 
activities related to the OnLine at PA Libraries project. The report is not intended to 
serve as a history of the project, project activities, or specific levels of involvement 
and work by various project libraries or other participants. The report dociunents the 
success of the project and can serve as a discussion point for all participgints as to (1) 
the successes and problems that arose with the project, (2) key issues that will 
require additional attention, and (3) planning strategies to continue the effort and 
successes of public libraries connecting the people of Pennsylvania to the Global 
Networked Society. 
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SURVEY AND DATA COLLECTION FINDINGS 



The consultants conducted four surveys as part of the OnLine at PA Libraries 
evaluation project: 

• One participating library survey in March 1997 to assess the participating 
libraries’ expectations and issues concerning the OnLine at PA Libraries 
project (see Appendix B); 

• One user survey in May 1997 to determine user satisfaction, demographics, 
and overall project/Internet assessment (see Appendix C); and 

• Two training evaluation surveys, one each for the initial training session in 
October 1996 (see Appendix D) and the follow-up training session in April 1997 
(see Appendix E). 

This chapter details the methodologies used to conduct the surveys and presents 
survey findings. 



PARTICIPATING LIBRARY SURVEY 

The participating library survey was distributed to all 188 participating library 
directors in March 1997. Of the 188 surveys, 144 were returned for a 76.6% response 
rate. Office of Commonwealth Libraries staff, SIU staff, and selected librarians 
reviewed and commented on preliminary survey drafts. These comments and 
suggestions were incorporated into the final survey form (see Appendix B). SIU and 
Office of Commonwealth Libraries staff assisted with non-responding library follow- 
ups to encourage participation. 



General Findings 

As of March 1997, participating library OnLine at PA Libraries connections were 
operational for an average of 4.7 months, with connectivity ranging from .5 to 8 
months (see Figure 2-1). The average monthly connection charge was $30.15 per 
month, with monthly costs ranging from $0.0 to $350.00 (see Figure 2-1). 



Figure 2-1. OnLine at PA Libraries Library Connection Costs/Months. 
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Figure 2-2. Primary OnLine at PA Libraries Library Interest. 
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Library directors indicated that the top three primary motivating factors for 
participating in the OnLine at PA Libraries project was interest on the part of library 
staff with 42.5%, followed by community interest with 32.2%, and availability of 
computing equipment with 16.4% (see Figure 2-2). 

Overall, most libraries engaged in some formal advertising activities (e.g., stories 
in local papers, presentations to local groups) to promote the OnLine at PA Libraries 
project in their libraries. A vast majority of libraries had between one and five 
instances of formal advertising as of March 1997 with 82.6% (see Figure 2-3). 



Figure 2-3. OnLine at PA Libraries Library Formal Advertising. 



Number of Formal Advertising 


% Advertising 


0 times 


9.7% 


1-5 times 


82.6% 


6-10 times 


5.6% 


11-15 times 


0.7% 


16-20 times 


0.7% 


21 or more times 


0.7% 



Library Training/Users 

The survey asked library directors to indicate the number of formal user training 
sessions the library conducted as well as the average amount of training time new 
users received from library staff. As Figure 2-4 indicates, a majority of libraries — 
63.2% — did not provide formal user training sessions. On average, librarians spent 
12.1 minutes with new workstation users, ranging from 0 to 35 minutes (see Figure 2- 
4). 
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Figure 2-4. OnLine at PA Libraries Library Formal Training Sessions. 



Number of Formal Training Sessions 


% Training 
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25.7% 


6-10 times 


3.5% 


11-15 times 
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16-20 times 


2.1% 


21 or more times 
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Average initial help (in minutes): 12.1 minutes (range: 0 minutes - 35 minutes) 



Figures 2-5 and 3-6 provide an estimate of OnLine at PA Libraries workstation 
users. As Figure 2-5 indicates, there is an average of 15.6 children and students uses 
of the workstation per week, 16.3 adults uses of the workstation per week, and 10.4 
library staff uses of the workstation per week (note: one session may not necessarily 
correspond directly to one user; one user may account for multiple sessions). 



Figure 2-5. OnLine at PA Libraries Library Number of User Sessions/Week. 
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Figure 2-6 presents some estimates of the number of users of the workstations 
per week. These estimates have the following limitations: 

(1) They depict a one-use, one-user approach. This may not be the case, as one 
user may account for more than one use. 

(2) They account only for an estimate of OnLine at PA Libraries workstation 
users and do not extend to other Pennsylvania pubUc libraries providing public 
access to the Internet. 
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Figure 2-6. Calculating the Average Number of Users per Week.* 





Using the Average 
Number 


Using the 3rd 
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Children and Students (through age 18) 


2,933 


3,800 


Adults (over 18) 


3,064 


3,800 


Total 


5,997 


7,600 


* Issues: (1) Estimates based on use data collected in March 1997. 

(2) Estimates based on 188 participating libraries. 



(3) They use library director-provided estimates from March 1997. Other data 
collected as part of the evaluation project suggest that workstation usage 
since that time has increased significantly. 

With those limitations, an estimated average of 5,997 users use the workstations per 
week. By using the 3rd quartile, rather than the average, an estimated 7,600 users 
use the workstations per week (see Figure 2-6). 



Benefits and Issues 

Library directors consider the top three primary benefits of the OnLine at PA 
Libraries workstation to be the expansion of available reference resources with 
48.3%, followed by the attraction of new users to the library with 31.7%, the 
expansion of the scope of the overall library collection with 13.1% (see Figure 2-7). 



Figure 2-7. OnLine at PA Libraries Library Benefits. 
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Figure 2-8. OnLine at PA Libraries Library Assessment of the Internet and 

OnLine at PA Libraries Connection. 
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2.3 


Library staiff have very good knowledge and skills 
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2.8 
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1.9 
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accessing and reading pornography via the public 
access Internet workstation 


4.5 


Generally, the comments from the users of the public 
access Internet workstation are very positive 


1.7 


We have had very few technical difficulties with the 
equipment provided by the grant 


2.3 


l=Agree 5=Disagree 



As Figure 2-8 indicates, the library directors rate the OnLine at PA Libraries 
project highly along a number of dimensions - library visibility (2.3 out of 5, with 1 
being “Agree”), collection expansion and augmentation (1.9 out of 5, with 1 being 
“Agree”), and user impressions (1.7 out of 5, with 1 being “A^ee”). Of particular 
interest is that library directors indicate no problems with users accessing 
pornographic material (4.5 out of 5, with 5 being “Disagree”). 

Finally, although library directors view the OnLine at PA Libraries project as a 
success, few could afford to access the Internet without the project (see Figure 2-9). 
A vast majority - 88.9% - indicated that they would have been willing to pay for the 
workstation, but could not afford to do so. 
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Figure 2-9. OnLine at PA Libraries Library Purchase on Own. 



Types of Users 


% Purchase 


Yes 


2.7% 


Yes, but imable to afford without the Bell 
Atlantic grant 


88.9% 


No 


2.0% 


Not Sxire 


6.2% 



USER SURVEY 

As part of the evaluation study, the consultants conducted a user survey in all 
participating libraries. The survey sought to obtain (1) an estimate of the number 
and types of users accessing the Internet via the OnLine at PA Libraries terminals in 
the participating libraries, (2) a description of that use, i.e., the types of resources and 
services users contacted, (3) an assessment of the usefulness of those resources and 
services contacted, and (4) an assessment of the usefulness of the OnLine at PA 
Libraries project as a means of access to the Internet. 



Background 

During a sample one week period in May, all 188 participating libraries assisted 
the investigators to collect data regarding the public’s use of the Internet. Prior to 
that sample week period, a packet of information that included instructions and data 
collection forms were distributed to participating libraries at the OnLine at PA 
Libraries spring training sessions. Participants were also informed of the purpose, 
intent, and importance of the user survey to the OnLine at PA Libraries assessment 
project dxiring these training sessions. 



Data Collection Instrument 



The consultants developed the two-page survey (see Appendix C) as a means for 
OnLine at PA Libraries workstation users to provide data concerning (1) their general 
familiarity with the Internet, (2) the utility of selected Internet-based resources, and 
(3) their impressions of the Internet in general and OnLine at PA Libraries project in 
particular. 



Selected OnLine at PA Libraries project staff reviewed the initial survey. Based 
on comments from that review, the consultants revised the survey. The revised 
survey was then pre-tested by two MLS students at the School of Information 
Studies, Syracuse University. Comments and feedback from this pre-test were then 
incorporated into the final survey. 



McClvire and Bertot 



October 1997 



20 



Evaluation of the OnLine at PA Libraries Project 



Survey Distribution Technique 

During the one-week sample period in May, all participating libraries asked up to 
25 users of their OnLine at PA Libraries workstation to complete a survey. A 
librarian or volunteer gave the users the survey, explained to them the purpose for 
collecting the data, and instructed the users in the completion of the survey. 

The librarian or volunteer instructed the users to complete Part I of the survey 
prior to accessing the Internet, complete Part II while connected to the Internet, and 
complete Part III after their Internet session. At the end of the sample week or 
having completed 25 surveys (which ever came first), all completed questionnaires 
were returned to the investigators for analysis. Of the 4,700 distributed surveys, 
1,925 usable surveys were returned for a 41.0% response rate. 

User Demographics 

As Figure 2-10 shows, slightly more females (51.8%) use the workstations than 
males (48.2%). The average user age is 29.1, ranging from 10 to 81 years (see Figure 
2-10). A majority of users - 56.6% - have a high school or elementary education 
level, with the remaining 43.4% having a community college or higher level of 
education (see Figure 2-11). Readers should keep in mind that a substantial 
percentage of users - approximately 35.3% - are elementary and high school 
students. 



Figure 2-10. OnLine at PA Libraries Users. 
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Figure 2-11. OnLine at PA Libraries User Education. 
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Users and the Internet 

The OnLine at PA Libraries workstation attracted a substantial percentage - 
26.0% -- of new users to participating libraries (see Figure 2-12). Of all the users, 
47.6% rate themselves as beginner users, 8.3% as intermediate users, and 44.0% as 
expert users of the Internet (see Figure 2-13). As Figure 2-13 also indicates, the 
typical user had an average of 8 months’ experience in using the Internet and had 
used the library’s OnLine at PA Libraries workstation 5 times in the past month. A 
vast majority of users had not formal Internet training with 64.7%, while 21.9% of 
users had between one and three hours of formal Internet training, and just 13.4% 
had more than three hours of formal Internet training (see Figure 2-14). 



McClure and Bertot 



October 1997 



22 





Evaluation of the OnLine at PA Libraries Project 



Figure 2-12. Percentage of New Users Due to 
OnLine at PA Libraries Program. 
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Of particular interest is that 34.5% of the OnLine at PA Libraries workstation 
users have no other Internet access means other than through the public library (see 
Figure 2-15). It is noteworthy, however, that 63.5% of the workstation users do have 
other means with which to access the Internet and continue to use the library’s 
Internet connection. 

Overall, users found their experiences with the OnLine at PA Libraries 
workstation and Internet to be positive, with ratings of their experiences ranging from 
1.7 to 2.4 (with l=Very Much So and 5=Not At All) (see Figure 2-16). Of particular 
interest is that users found the information they wanted on the Internet and found it 
to be useful, found the equipment/workstation easy to use, and would tell other 
members of the community about the library’s Internet connection. It is also 
noteworthy that the users’ ability to use the eqioipment and find the information that 
they wanted via the Internet was not affected by the level of formal Internet training 
that users had (see Figure 2-17). Indeed, users were able to make similar use of the 
workstation and the Internet regardless of the amormt of formal training. 
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Figure 2-13. OnLine at PA Libraries User Expertise. 
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Figure 2-14. Formal Internet Training of Users. 
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Figure 2-15. OnLine at PA Libraries User Other Access to the 

Internet. 



34 . 5 % 
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Figure 2-16. OnLine at PA Libraries User Assessment of the Internet and 
OnLine at PA Libraries Connection. 



Internet Assessment 


Average Assessment 


While on the Internet, I foimd the information that I came 
to the library to obtain 


2.2 


While on the Internet, I foimd information that was very 
useful and interesting 


1.9 


The Internet information I located will assist me greatly 


2.2 


The Internet access provided by the library is relatively 
easy to use 


1.8 


Having access to the Internet via the library is very 
important to me 


1.7 


The computer and telecommunications equipment all 
worked very well 


1.8 


I found the speed of the Internet, e.g., loading graphics and 
downloading information, to be quite good 


2.4 


I would tell other members of the community about the 
librarys Internet connection 


1.7 


I was able to use the Internet without library staff or other 
assistance 


2.0 


l=Very Much So 5 


=Not At All 
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Figure 2-17. OnLine at PA Libraries User Assessment of the Internet and 
OnLine at PA Libraries Connection by Amount of Training. 



Internet Assessment 


No 

Training 


1-3 Hoius 
Training 


3+ Hours 
Training 


Overall 

Assessment 


While on the Internet, I found the 
information that I came to the 
library to obtain 


2.2 


2.2 


1.9 


2.2 


The Internet access provided by the 
library is relatively easy to use 


1.8 


1.7 


1.6 


1.8 


I would tell other members of the 
community about the library’s 
Internet connection 


1.7 


1.6 


1.5 


1.7 


I was able to use the Internet 
without library staff or other 
assistance 


2.2 


1.8 


1.5 


2.0 


1 =Very Much So 5=Not At AU 
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TRAINING SURVEYS 



The consultants conducted two training surveys. The first survey occurred during 
the initial project training session in October 1996 (see Appendix D), and the second 
during the advanced training session during April 1997 (see Appendix E). The 
purpose of the surveys was to determine the: 

• Extent to which the training sessions met the needs/anticipated needs of the 
attending librarians; 

• Abihty of attending Hbrarians to apply training instruction in their libraries; 

• Ability of attending librarians to begin and/or enhance ongoing Internet training 
activities in their libraries; and 

• Overall impression of the OnLine at PA Libraries project in training 
participant libraries. 

Together, these data serve to inform the project’s training process for future training 
sessions. 

Initial Training Session 

All participating libraries had to send one participant to the initial training session 
in October 1996. This training session provided attendees with instruction on:^ 

• Workstation configuration and general workstation operation; 

• How to set up the workstation in the library; 

• How to connect to an Internet Service Provider (ISP); and 

• General Internet navigation. 

The treiining session lasted one day. 



The initial training session began prior the commencement of the evaluation 
project. Thus, surveys could only be collected from participants at the latter training 
sessions. The consultants used the training lesson plan and documentation to 
develop a draft survey. Project staff provided feedback on the draft survey, and 
changes were made to the draft survey to reflect staff comments and suggestions. 
Each attending representative received a copy of the training evaluation form at the 
end of the training session. Since only a portion of the training sessions received the 
survey, a total of 55 completed surveys were returned. 
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Findings 

Of the attending trainees, 50.0% were professional librarians with MLS degrees, 
and 50.0% were non-professional library staff (sample titles included clerks, library 
administrator, and volunteer). The attendees had an average of 11.4 years 
experience, ranging from 1 year to 24 years. 

Overall, attendees rated their Internet navigation skills as “below average” with 
43.6%, followed by “average” with 38.2%, “above average” with 16.4%, and “excellent” 
with 1.8% (see Figure 2-18). Attendees rated their computing skills slightly higher 
than their Internet navigation skills with 43.6% indicating “average” computing skills, 
29.1% “below average” computing skills, 25.5% “above average computing skills, and 
1.8% “excellent” computing skills (see Figrure 2-18). 



Figure 2-18. Initial Training Participant Internet and Computer Expertise. 



Expertise Rating 


Internet Navigation 
Skills 


Computer Use 
Skills 


Below Average 


43.6% 


29.1% 


Average 


38.2% 


43.6% 


Above Average 


16.4% 


25.5% 


Excellent 


1.8% 


1.8% 



Attendees strongly agreed that there is significant interest in the OnLine at PA 
Libraries project at their respective libraries (rating of 4.4, with l=Strongly Disagree 
and 5=Strongly Agree). Similarly, attendees strongly agreed that having access to 
the Internet can be useful for riiral libraries (rating of 4.7, with l=Strongly Disagree 
and 5=Strongly Agree) (see Figm-e 2-19). 
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Figure 2-19. Initial Training OnLine at PA Libraries Project and Internet 
Assessment. 



OnLine at PA Libraries/Intemet Assessment 


Average 

Assessment 


There is a great deal of interest in this project at my library 


4.4 


Internet-accessible resources can be useful in a rural 
library context 


4.7 


l=Strongly Disagree 5=Strongly Agree 



Figure 2-20. Evaluation of Initial Training Components. 



Training Component 


Beginning 
of Day 


End of 
Day 


Ability in 
Library 


URL Addressing 


2.7 


3.8 


3.7 


Netscape Navigation 


2.6 


3.8 


3.8 


Dialing Internet Service Provider 


2.4 


3.7 


3.7 


Setting Up PC/Printer 


2.6 


3.9 


3.8 


Printing Documents 


3.3 


4.0 


4.0 


l=Not At AU 




5=A Great Deal 



As Figure 2-20 shows, the training session successfully assisted attendees gain 
the necessary skills to set up the OnLine at PA Libraries workstation, connect to the 
Internet, and make basic use of Web-based resources. 

Overall, attendees rated the training session highly (see Figure 2-21). Indeed, 
attendees found that the training met their goals, the presentation of the material 
was clear, the trainer was well prepared, and that they had a basic sense of how the 
Internet works (ratings ranging from 3.7 to 4.5, with l=Not At All and 5=Very Much 
So). Attendees indicated some concern over the amount of material covered during 
the session (rating of 2.3); however, as comments and the self-assessments indicated, 
attendees possessed var 3 dng levels of computer/Intemet sophistication. The training 
session was targeted to provide only basic Internet and computing skills, thus some 
attendees found the material relatively easy. 
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Figure 2-21. Initial Training Overall Assessment. 



Internet Assessment 


Average 

Assessment 


My goals for this training session were met 


3.8 


The ideas presented in this training session were very clear 


4.0 


There was sufficient opportunity to ask questions during the training session 


4.5 


Overall, the trainer(s) was well prepared 


4.5 


There was too much material covered in this training session 


2.3 


I feel like I now have a good basic imderstanding of how the Internet works 


3.7 


Please estimate the degree of difficulty you will have in configuring the 
hardware/software for this project when you return to your library 


2.7 


l=Not At All 5=Very Much So 



Spring Training Session 

OnLine at PA Libraries project staff designed the spring training session, 
conducted in April 1997, to provide advanced instruction on managing Internet-based 
resources for library use. In particular, the training session covered such topics as; 

• Using Web-based search engines; 

• Constructing appropriate searches for particular Web-based search engines; 

• Identifying and using Web-based reference resoiorces; 

• Downloading files and software from the Web; and 

• Using Fortress, the security program loaded on each OnLine at PA Libraries 
workstation. 

OnLine at PA Libraries project staff designed the training topics based on input from 
the participating libraries. 
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Methodology 

Each participating library was able to send up to three staff members to the 
training session. Training occurred in multiple locations — with adequate training 
facihties — so as to minimize travel for library staff. 

The consultants used the training lesson plan, documentation, and discussion with 
project staff to develop a draft survey. OnLine at PA Libraries project staff provided 
feedback on the draft survey, and changes were made to the draft survey to reflect 
staff comments and suggestions (see Appendix E for a copy of the survey). 

Each attending representative received a copy of the training evaluation form at 
the end of the training session. A total of 368 completed surveys were returned. 



Findings 

The time lapse between training sessions, which allowed for librarians to develop 
familiarity with the Internet, had an impact on the Internet expertise of attendees. 
Overall, attendees rated their Internet navigation skills as “average” with 53.2%, 
followed by “below average” with 26.5%, “above average” with 18.9, and “excellent” 
with 1.4% (see Figure 2-22). Thus, as compared to the initial training session (see 
Figure 2-18 above), more attendees had greater Internet expertise. 

As Figure 2-23 indicates, the training session was particularly successful in 
instructing attendees on how to search the Web, use Web-based search engines, 
access government documents on the Web, identify/use reference resources on the 
Web, and create and organize bookmarks. The session was less successful in 
providing attendees with instruction on downloading Web files, downloading Web- 
based software, and using Fortress (the workstation security program). 



Figure 2-22. Spring Training Participant Internet Expertise. 



Internet Navigation Skills 


Percentage Skill 


Below Average 


26.5% 


Average 


53.2% 


Above Average 


18.9% 


Excellent 


1.4% 



ERIC 
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Figure 2-23. Evaluation of Spring Training Components. 



Training Component 


Beginning of 
Day 


End of Day 


Searching the Web 


2.9 


4.0 


Using Web-based search engines 


2.7 


3.9 


Accessing government documents on 
the Web 


1.8 


3.4 


Identifying/using reference resources on 
the Web 


2.4 


3.7 


Creating and organizing bookmarks 


2.3 


3.7 


Downloading Web files to a diskette 


1.7 


2.7 


Downloading software fi-om the Web 


1.7 


2.5 


Using Fortress 


1.6 


2.8 


l=Not At All 


5=A Great Deal 



Figure 2-24. Spring Training Overall Assessment. 



Internet Assessment 


Average 

Assessment 


My goals for this training session were met 


1.9 


The ideas presenting in this training session were very clear 


1.7 


There was sufficient opportunity to ask questions during the 
training session 


1.5 


Overall, the trainer(s) was well prepared 


1.4 


There was too much material covered in this training 
session 


3.5 


I feel like I now have a good basic imderstanding of how to 
use the Internet as a reference tool 


1.9 


I will now be able to use the Internet to answer reference 
questions 


2.0 


The Internet will be a welcome addition to our library’s 
reference tools 


1.4 


l=Strongly Agree 5= 


=Strongly Disagree 
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Overall, the attendees rated the spring training session highly in all aspects (see 
Figure 2-24). Attendees found that the trainer was well prepared, the course material 
was presented clearly, participants were able to ask questions, and the course had 
the right amount of content. Perhaps more importantly, participants felt that the 
training would enable them to go back to their libraries and use the Internet as a 
reference tool. 

The training also provided attendees with the necessary skills, desire, and 
willingness to begin and/or engage in increased user/librarian training activities. In 
particular, attendees indicated that they would provide training to other library staff 
(76.4%) and the public (70.9%). 

Finally, attendees of the spring training session rated the OnLine at PA Libraries 
project as “very successful” (1.0, with l=Very Successful and 5=Very Unsuccessful). 



HELP DESK LOGS 

The study team reviewed the help desk logs maintained by SIU staff from October 
1996 through June 30, 1997. SIU staff designed the help desk logs (see Appendix F) 
to: 



• Track problems and issues that participating libraries faced while using the 
project equipment; 

• Track problems and issues that participating libraries encountered while using 
the Internet; 

• Determine what, if any, trends in calls that the SIU received so as to design 
future training programs and interim help memos to distribute to participating 
hbraries; 

• Serve as a liaison between participating libraries and service technicians, thus 
be able to determine the necessary support for participating libraries -- SIU or 
other; and 

• Plan for future project needs. 

The logs, therefore, provided the SIU and other project managers with valuable 
feedback on the OnLine at PA Libraries project’s implementation. 

When the SIU received a help desk call, an incident report (log) was initiated. The 
problem was identified, described, and attempted to be resolved immediately. If 
immediate resolution was not possible, the incident report was considered open until 
formal problem resolution was attained. 
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Overall, a review of the help desk logs show the following trends: 

• Calls received at the beginning of the project involved hardware/software set 
up and configuration issues, problematic hardware (e.g., bad monitors, faulty 
modems), and establishing an Internet connection; 

• Calls received once the PCs were up and running expanded from set up to 
usage issues — e.g., how to configure and use Netscape; and 

• Although relatively few, some calls indicated the need for periodic PC 
reconfiguration to the original settings due to user-made modifications to the 
system configuration and librarian error. 

These trends reflect the cycle of systems implementation issues — from 
implementation to general usage. 



SUMMARY 

The data from the various surveys and help desk log file analysis indicate that the: 

• OnLine at PA Libraries program is highly successful; 

• Internet serves to increase the scope and range of library collections and 
reference material; 

• Librarians are better able to answer patron questions; 

• Public holds the libraries in higher regard due to the workstations, and wants 
an expansion of the project through additional workstations, training, and 
facilities; 

• Help desk log files can assist project managers determine project hardware, 
software, and configuration usage issues; and 

• Training sessions (initial and spring) were successful in providing attending 
library staff with the skills necessary to install and use the workstation as well 
as how to best use the workstations for improved services delivery. 

These data suggest that based on a number of different criteria, different surveys, 
and responses from different tjqjes of individuals, i.e., library directors, library staff, 
and the public, the OnLine at PA Libraries project accomplished its objectives and 
was very successful. Indeed, data presented in the next chapter that describes the 
site visits reinforces the findings presented from the surveys reported in this chapter. 
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SITE VISITS 



One key component of the data collection activities for the OnLine at PA Libraries 
project evaluation effort was site visits to four libraries participating in the project. 
The site visits provided the evaluators with: (1) an opportunity to meet with the site’s 
library directors; (2) site library staff; (3) library directors and staff from surrounding 
libraries; and (4) users of the public access Internet service provided at the site 
libraries. This Chapter provides background information regarding the method 
employed for the data collection effort at the sites and a summary of the findings 
from each site. 



INTRODUCTION 

The investigators conducted the site visits during the months of June and July 
1997. The objectives of the site visits were to: 

• Observe the set-up and use of the public access Internet workstation and 
connection; 

• Meet with the library directors, staff, local area librarians, and users to obtain 
their comments and assessment regarding project activities and the use of the 
public access Internet workstation and connection; and 

• Identify factors that affected the overall success of providing public access to 
the Internet via the OnLine at PA Libraries project. 

These objectives guided the development of the data collection activities employed for 
the site visits. 

In discussions with officials from the Office of Commonwealth Libraries, the 
investigators developed the following set of criteria to select site libraries: 

• Geographic region. To obtain geographical representation throughout the 
state, the investigators defined four regions. The regions were the northeast, 
southeast, southwest, and northwest/central. 

• Rural designation. A key component of the OnLine at PA Libraries project 
was to provide public access to the Internet in rural areas of Pennsylvania. An 
emphasis on rural libraries, through population density data, was therefore 
critical. 

• Workstation. Site libraries had to have at least one OnLine at PA Libraries 
workstation that had an Internet connection as of February 1997. 

• “Representative” library. Each site library needed to be a “typical” library 
within the region. 
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Candidate libraries within these regions were identified within the geographic regions 
using these criteria with assistance from the Office of Commonwealth Libraries staff. 

During May and June 1997, the investigators contacted the directors of the 
libraries to confirm dates for the one-day visit. They sent each library director a 
letter detailing the schedule and proposed activities for the visit. The letter described 
topics that would be discussed in the various data collection efforts during the visit 
(see Appendix G). Each site visit included the following data collection activities: 

• Selected documents and library background information provided to the 
investigators prior to their visit as well as collecting additional items while at 
the hbrary; 

• Interview with the director of the library; 

• Group interview with library staff who had been involved in the planning or use 
of the public access Internet workstation; 

• Focus group session over lunch with near-by library directors who also were 
participants in the OnLine at PA Libraries project; 

• Focus group session with users from that library of the public access Internet 
workstation; and 



• Follow-up phone conversations with the directors of the library (when needed) 
for additional information or clarification of the information obtained. 

All libraries were extremely helpful and cooperative in assisting the investigators 
organize the site visits. 

At each library typically there were two to four library staff interviewed; five to 
nine neighboring hbrary directors interviewed; and six to 13 users in each of the focus 
groups. The investigators asked all participants and users to complete a 
questionnaire that obtained background information as well as their assessment of 
selected key issues (see Appendix H and I). The library directors were especially 
successful in obtaining participation from library users for the focus group sessions. 

Both investigators participated in the site visits for libraries A and B. Then each 
did one additional library - C and D. This strategy was important as a means for 
the investigators to fine tune the techniques and instruments during the first two 
visits together. Information from each of the visits informed later ones as to 
discussion topics and techniques. Each investigator developed notes from the sites 
which were then organized into this report that summarized findings for each library. 

Thus, the following four summaries for each of the libraries are not verbatim 
accounts of the various data collection activities as described above. Rather, the 
investigators have organized the content into key topics and reported them below. A 
significant amount of information from lengthy notes, the questionnaires, and 
background information has been distilled into the following summary. 
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LIBRARY A 

The library’s budget is $120,000 with approximately $20,000 for books and 
$1,500 for periodicals. The library is located in a county with a population of 47,800; 
the population served is 27,000, but it also serves a significant number of individuals 
from around and outside the county. The facility was renovated in 1987 and has 
about 4,000 square feet. Only 4% of the county residents have a college degree. 
There are four Full Time Equivalents (FTEs), one part time and one student; the 
library is open 63 hours per week. The library works with two library boards - a 
county governing board and a system board that is more advisory in nature. The 
area is economically depressed although there are some small businesses. 
Agriculture is an important component of the local economy as well as “traffic from 
the Thrnpike.” 



Interview with Director 

Initially the director was reluctant to jump on this project due to his perceived lack 
of technical sophistication and awareness of computers in the beginning of the 
project. Further, the library had very tight resources, limited staff (who were not 
knowledgeable about computers), and limited time. At the time of the site visit he 
was very impressed with the workstation and provision of public access Internet use. 
He was very glad that this library had been part of the project as Internet access 
would not have occurred without such participation. And, he thought that the 
amount of time and effort dedicated to this project was not as extensive as he 
originally thought. 



Set-un and Connectivity 

The library received the workstation in October 1996; there was a 2-3 week delay 
in obtaining an Internet Service Provider (ISP) and phone line. The actual set-up was 
not a problem. The initial ISP went out of business; now they have a local provider 
who charges $ 19.95/month; the phone line is $30/month; and they spent $200 for 
installation. ($760 per year). The library has not explored leased line possibilities as 
yet and was unaware of leased line as an option for better and faster connectivity. 



Library’s IT Infrastructure 

The library has four PCs — OnLine PA; Access PA; and two OPACs that connect 
to the server. The director was very interested in how a LAN/networked approach 
might be developed rather than the “stand alone” approach currently in place. He 
would welcome some training from the DLC or other sources on IT planning and how 
to maximize impact of computing in the library and on services. One of the 
investigators explained benefits of a networked strategy and he was very interested 
in such applications in this library. He said he would ask the DLC for workshops and 
information on IT planning and networks. 
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Support for Information Technology 

The bottom line is that there is no money available for computers and information 
technology either in the library budget or from new sources in the coimty. The library 
was $8000 short in revenue this year which, if not for a fund raising effort that 
covered most of the shortfall, would have been taken out of the book fund. The 
director is interested in advice and planning information for what to do next to support 
computing efforts at the library given this situation. He is well aware of the need for 
upgrades and additional equipment but not sure how or if this can be done without 
state aid. 



Persnective of the Boards 

The coimty board was pleased with the project and the library’s participation in it, 
although such participation and increased county visibility did not translate into more 
resources. The system’s board has little involvement in the project. By and large 
board members are not familiar with the Internet or computers. The director would 
like the board members to become more knowledgeable about the public access 
workstation. 



Staff Interest 

Generally, the staff were not technologically literate about computers and had 
some significant hesitation about using the public access workstation. The director 
has probably been the heavier user of the workstation although staff are now 
“beginning to come around.” Some of the staff thought the workstation would be “an 
albatross” and require too much time and effort. They were also concerned about the 
use of the computer to access pornography. Staff, for a variety of reasons, have yet 
to attend formal training sessions on the Internet or computers in conjunction with 
this project. 



Visibility and Credibility 

The director believes that the public access terminal has been helpful in raising 
the overall visibility/credibility of the library within community. He thinks that the 
workstation’s presence directly helped with this year’s annual fund drive which raised 
$8000 rather than the usual $2500-$3000. It is clear that a number of library users 
are pleased and impressed with the new service. He also noted that change comes 
slow in this county and that significant visibility of the public access service will take 
time. 
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Users 



The director believes that Internet users are more than 50% male while the usual 
library user is 80-85% female. Internet users are usually “middle class, business 
professional” which is a group that does not normally come into the library . There is 
little use by the high school although it is across the street. Those students that do 
use the workstation are the “responsible ones.” Apparently there are few terminals 
for the students in the schools currently, but this will change in the next year as some 
access and training will be provided. The director thinks that for some workstation 
users, they come to the library to use it for a while, are impressed, go and then try to 
get a connection at home. 

There has been only one problematic user. He stayed on all day using chat rooms, 
when asked to cut back he got upset and wrote a letter to his state representative 
complaining about being denied access. There have been no other problems, but they 
intentionally put the terminal within view of librarians and close to the main desk. 
Initially they had a “no one under 18” policy without signed parental consent, but 
dropped the requirement due to the responsibility of students that were using the 
terminal. The library is considering a new policy/parental consent form. 



Services 

The director provides assistance to users “as best I can.” He believes that the 
staff are “coming around” to providing assistance although there are some elderly 
staff who will probably never learn to use the computer and especially the mouse. 
Most of the users are self taught and get “minimal” assistance from the library staff. 
The director may have been too modest about his involvement as later in the user 
focus group participants regularly complimented him for all of his time 
assistance in using the workstation for reference help. The director indicated that he 
has developed his own set of bookmarks for the patrons. He stated that he sees 
himself as competent enough to get the average user on the right track, but may not 
be able to help more sophisticated users. 



Training 

The director attended the first OnLine at PA Libraries project training session and 
was not impressed with it or the training. He left with the impression to not call for 
help if needed. He was very frustrated by not being able to obtain the password to 
Fortress to change settings for display colors, resolution, modem settings, etc. He did 
comment that the handouts were good. The second session he enjoyed, learned a 
great deal, and found the focus on reference very helpful. The instructors at this 
session were much better, in his view, than those at the first session. His staff, 
however, would not attend. The director provided them with copies of handouts and 
some instruction from the session. He was not sure how much the staff read or used 
the handouts. He thought there was too great a lag time between the first and second 
session. 
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Impacts 

There have been a number of important impacts from the library’s participation 
in this project. First, the director sees himself as much more technically literate and 
interested in technology applications and uses in the library. He is very impressed 
with the public access Internet service and sees huge potential for its use as a 
reference tool and for other services. He hopes that the workstation’s presence will 
also be a catalyst to move staff into greater interest and use of computers and 
technology. 

In addition, he has received numerous comments from library users as to the 
importance and success of the workstation. Despite some recent difficulties with 
their service provider, there is constant use of the workstation. Some of the users 
have “congratulated” him for having the public access terminal as it is the only 
means by which they would be able to access and use the Internet. 

Finally, he thinks the public access terminal, although not directly helpful in 
obtaining additional funds from the county, did help him in fundraising and will help in 
the future. The “aura” of having the public access terminal makes the library look 
like it is “really in the information age.” He plans on future fundraising efforts to take 
more advantage of the “presence” of the computer. The long term issue of raising 
expectations with the presence of public access to the Internet without adequate 
funding and technology knowledge has him somewhat concerned. 



Staff Interviews 

The investigators met with two full-time staff. One had worked for the library 20 
years, the other for two years. When the library applied to participate in the project 
they had “mixed feelings.” They were skeptical as to “who would want to use the 
Internet in this county?” They were a bit fearful about having to learn how to use it 
themselves and being able to show how to use the workstation to others. 

They have started using the workstation, but they characterized themselves as 
“slow to learn.” Patrons started coming in and taught them and the director how to 
use the workstation and the websites. The two staff indicated that they really rely 
on sophisticated users to learn from, ifthey are willing to show them. Guides that 
came with the computers were problematic; hands-on use was better. 



Publicity 

There have been a few articles in the local paper about the new service. There has 
also been a story in a local Vo-Tech newsletter. Generally they believe that the 
publicity has come from word of mouth among library users. They thought more 
publicity was being developed. 



McClure and Bertot 



October 1997 



42 



Evaluation of the On Line at PA Libraries Project 



Users 



They thought that the users represent a diversity of ages — young to old, but 
mostly under 55 and between 25 and 50. They believed that the users were split 
relatively evenly between men and women and that they came from all economic 
levels, but predominantly blue collar -- which is the nature of the county. They 
thought that some have Internet access at home , but that the library’s connection 
was “better” or maybe “faster.” They also believe that the public access to the 
Internet attracted new users who had not previously used the library. “More than 
half of the users just use the Internet; they don’t check out books.” Use of the 
workstation continues to increase and as the students become aware of the access 
and learn about the Internet at school, use will probably “take off.” 

Except for the one instance with someone using the chat rooms extensively (see 
above) there have been no problems with improper use of the workstation or with 
pornography. The staff believe that with the workstation so close to the main desk, 
people would be too embarrassed if “caught viewing porn.” If kids begin using the 
workstation a lot, however, problems might increase. 



Assistance to Users 



In terms of assisting users, they try to help as best they can; they give a brief 
tutorial - but they noted that they only have limited knowledge and limited time to 
provide such tutorials. The amount of help they can provide someone really depends 
on how busy the library is and the availability of someone who knows enough to help 
the user. These staff begin work at noon on Mondays so the director is there alone to 
help that morning. They also rely on him for “more advanced help.” One staff 
member commented that generally it is the younger people who know how to use the 
workstation. They both recognized that they needed more training and confidence in 
using the workstation but were unsure how they would get it or become “expert” at 
the computer. 

Some patrons continue to offer suggestions for changes in the set-up of the 
computer - including settings on the PC. The initial password protection was an 
issue that frustrated them: couldn’t change things, but they were not technically 
competent to do so. They eventually relied on some users to help with configuration 
changes. 



Problems 



The public access creates problems for the librarians because the librarians 
cannot get access to the workstation and use it for reference questions/sources since 
it is usually being used by the public. Without a workstation that they can use 
regularly, they believe that their “comfort level” with the terminal will not increase. 
Both felt that they needed an additional terminal at the reference desk just for 
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librarians. It is a bit embarrassing that the patrons know more about the 
workstation and the Internet than the staff. 

They noted that the ISP problems in obtaining reliable connectivity created 
problems for them and users. A number of uses were quite frustrated in not being 
able to use the workstation when they came to the library. The new provider seems 
to be much better. 



Training 

For a range of reasons, the staff had not been very interested in training -- “not 
enough time,” “too much work to do,” “computer was complicated,” etc. Since they 
did not attend the second project training session (although the director asked them to 
attend), they still felt some imease at using the public access workstation. One of the 
investigators noted that there were some easy things they could do to help 
themselves: 

• Sign up for time at the workstation to be sure that they had an opportunity to 
use the workstation and have ‘Tiands-on” experience. 

• Have regulau: staff meetings once a week (including the director) to discuss use 
of the workstation and “favorite” sites that were good for reference. 

• Purchase a couple of trade magazines like Internet World, or NetGuide, to help 
them learn about good websites and other information about the Internet. 

• Take advantage of local training opportunities. 

They agreed that these were good ideas. But it is unclear if the strategies would be 
implemented. 



Conclusion 



The two staff interviewed were generally positive about the provision of public 
access to the Internet in their library. It probably has been a “good thing” for the 
county. Both circled “3” (1 = unsuccessful, 5 = very successful) when assessing the 
success of the OnLine at PA Libraries project on their questionnaires. They were 
unsure about how it might be integrated into traditional hbrary services and how they 
might better use the workstation. Throughout the session they noted their need to 
better learn how to use the workstation and provide better reference service, but they 
seemed to have some hesitancy about developing those skills. 
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Luncheon Focus Group with Neighboring Librarians 

Over lunch, the investigators met with five neighboring librarians who had 
participated in the OnLine at PA Libraries project and the director of the library at 
the site visit (six librarians total). Their budgets ranged from $34,000 to $85,000. 
The populations of their commrmities ranged from 4,500 to 13,000. They described 
their communities as “remote, rural, and to some degree, economically depressed.” 
Some of the libraries were open less than 40 hours per week and were “one or two- 
person” libraries. 

These were especially dedicated librarians. Many worked hours at the library well 
beyond that for which they were paid. All of the librarians were enthusiastic about 
the project and pleased to discuss their involvement in it. Their general view of the 
project was “its GREAT,” “the State Library did an outstanding job working with Bell 
Atlantic to do this,” and “people love it and appreciate it.” 



Set-up and Connectivity 

These librarians reported no significant problems with the set up of the 
workstation although one eventually had problems with the monitor. It took over a 
month to successfully fix the monitor and was quite a fhistrating experience. Three of 
the librarians reported having problems with the ISP and having reliable 
connectivity. Problems included two ISPs going out of business, poor reliabfiity, and 
the connection just going down unexpectedly. Costs for the connectivity were 
reasonable - usually $20-$30 per month although some indicated that it could be a 
problem for their budget. 



Users 



There was some agreement that most of the users were younger folks as well as 
students. One library was in a school district where students had some access to the 
Internet and thus, knew how to use the workstation. This library had heavy use of 
the public access terminal by kids. They all expect increased use by kids as the 
schools provide Internet access and training. Some thought that there would be an 
“avalanche” of students this summer. 

There was some variance across these libraries in terms of the nature of the 
users. In some instances new users came to the library as a result of the Internet 
access. In other instances the Internet users were the same users that always came 
to the library. One explanation for the possible difference was that some of the 
libraries had made a number of efforts to publicize the availability of the connection; 
these appeared to be the ones that then had “new” users. Most agreed that the users 
were often repeat users in that they used the Internet access regularly. 
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User Assistance 

A very real problem for these librarians was being able to provide user assistance 
and run the library at the same time. Oftentimes they were in the library by 
themselves and providing instruction on the use of the workstation could not be done 
given “a line at the reference desk and the phone ringing.” The level of assistance 
provided to users varied in light of how busy the librarians were with other tasks. 
Usually a short introduction to the workstation and basic URLs was all users needed. 
One librarian commented that the “hand holding” it took to get elderly folks to try the 
workstation and use it was “ridiculous.” And that even after that effort, the user 
gave up in firustration. 



Need for Second Workstation 

These librarians were emphatic about the need for a second terminal that was 
intended primarily for reference work and available only, or mainly, to the librarian. 
In most cases, it was very difficult for the librarian to use the terminal because 
others were on it. Plus, trying to learn how to use the terminal while with a user is 
difficult and oftentimes very embarrassing. They thought that if they were going to 
be good on the Internet they needed their own terminal. 

While they all were extremely impressed with the sources and information that 
one could get off various websites, they will not be able to take advantage of the 
workstation because users are on them “constantly.” Even when the librarians come 
in before or after library hours to learn the Internet, they then cannot access the 
workstation during regular operating hours because usually someone is at the 
terminal. There was some discussion about the project’s requirement that the 
workstation be a public access terminal rather than for the librarians. 



Success Stories 



All of these librarians had success stories concerning, finding resources or 
information that otherwise could “never have been obtained at the library.” One 
librarian was overwhelmed with the “neat stuff’ available from the federal 
government and noted that some of the statistical sources she had been using were 
“fantastic.” They also told of success stories from the users that ranged from one 
woman finding information about home gardening and plants to another that obtained 
agricultural information that helped him manage his farm. One librarian noted that 
they were almost always able to find material and print it for patrons that otherwise 
could not have been obtained except via interlibrary loan. Users were often “amazed” 
at what they could find. 
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Publicity 

The librarians had very different views about the need for and importance of 
publicity for the public access Internet workstations. One that had done some 
publicity noted that it did result in new people coming to the library, but that it also 
resulted in waiting lines and having to develop a sign-up sheet for when people could 
use the terminal. Another commented that she had “plenty of use and additional 
publicity was not needed” and any publicity may result in “too many people wanting 
to use the workstation.” 



Benefits and Impacts 

When the investigators asked the librarians to list what they thought were the 
most important benefits and impacts from the project, a flurry of responses resulted 
that included the following: 

• Vi.sibilitv and credibility . In these communities, users now saw the library as 
“at the cutting edge of using technology.” In addition, the librap^ is “the first 
place to try” to obtain information “because we have computers.” 

• Rftdnced intellectual isolation . People in the communhy now had “the same 
access to information and resources as people in the cities.” 

• Introduced computers to people . For a number of users in these libraries, this 
was their first exposure to a computer and learning how to use one. The users 
have been “empowered” to access information that they otherwise could not 
obtain. 

• Fxnanded and “cost free” collections . The librarians were overwhelmed with 
the range, quantity, and quality of reference sources that they (and users) now 
accessed. “My collection is now the entire world - and it costs me nothing,” one 
librarian commented. 

• TT.cie of e-mail . The ability of these librarians to now use e-mail to contact their 

district library and other libraries and individuals is very important to them. 
Because they cannot afford long distance calls, they can now “talk to anyone in 
the world.” i 

• Reduced Inter-library loans . These librarians believe that they are requesting 
fewer interlibrary loans. Users come in and “usually they can find something 
on the net that meets their need -- and it is full-text.” 

• Collaboration with schools . Two of the librarians noted that they were 
collaborating with the local schools with training and use of the workstations. 
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• Leveraged donations and f\indraising . As with the view of the director of the 
site library, the appearance of the computers seems somehow to have helped 
them in various types of fundraising efforts. One person commented that “new 
services seems to help fundraising -- especially if the services help children.” 
Another received a $25,000 donation that came, the librarian believes, at least 
in part, because of the new computer presence at the library. 

• Jnmp-.started the library and the librarians . These workstations quickly 
moved the libraries into the computer age and did it in a relatively short time. 
The project also got the librarians to become computer literate and for these 
librarians, “I am very interested now in using the computer in the hbrary.” 

In short, there were numerous impacts and benefits that have resulted from 
participation in the project. Some commented that there may be more benefits yet 
to be realized since they had only provided Internet access for 4-5 months. 



Too Succes.sful and Raising Unrealistic Expectations? 

To some degree, there was a sense that the OnLine PA project had been “too 
successful” in that the libraries had significant amounts of users and work that had 
not been there prior to the project ~ and she noted “no additional staff or resources for 
that workload.” One librarian commented that the success was wonderful, but she 
needed “more staff and resources to keep up” with the demands placed on the library. 
The project had raised community expectations for library services and use of 
computing. The reality, however, was that no additional resources from^^ local 
governments would be forthcoming and they wondered how they would “cope” with 
this success in the future. 



Recommendations 

There were two sets of recommendations. The first set of recommendations dealt 
with the OnLine at PA Libraries project directly: 

• The reference training (second session) needed to be done earher in the project. 
More “hands-on” training would have been good. Possibly the district libraries 
could be more involved in training and project support. 

• Not providing the libraries with the password to disable Fortress was 
unfortunate and just made it more difficult for the libraries to finally get the 
computers configured correctly. 

• Treat librarians in participating libraries as partners in the project, not 
“problem children.” While it was clear that not all library staff and directors 
were technology literate, most consider themselves quick studies. To not 
include the librarians in the decisions of computer configurations (e.g., what 
speed the modems were set to and the resolution of the video) seemed 
inappropriate. 
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• A district electronic discussion list or perhaps a list for all the participating 
libraries in this project to share information about what they have learned and 
successes would be very useful. 

• Clarifying the “next steps” for the libraries now that they are connected and 
providing public access Internet service. “What are we supposed to do now?” 

Another set of recommendations dealt with suggestions for “Phase 11” of the project: 

• Provide each district library with a projector that could be loaned throughout 
the district so that the librarians could demo what the Internet connectivity 
was “all about” for groups in the commrmity. 

• Hold more training sessions and better share information in the district about 
successes and how to best use the new workstation. 

• Provide each library with a second workstation that could be used primarily by 
the librarians for reference services. 

• Develop an on-going collaboration with Bell Atlantic or other organization to 
support projects such as this. 

• Learn more about IT planning and how to integrate the computers into the 
library better. 

These librarians felt strongly about the need for some continuation of the project and 
to have a “Phase II.” 



Conclusion 

Clearly the librarians thought the project was very successful. The average 
response on their questionnaires regarding the success of the project was 4.2 (1 = not 
successful, 5 = very successful). They were concerned, however, that this “quick fix” 
could not be carried through into the future. To some degree this project whetted the 
librarians’ appetites for more computing and Internet resources. But they believed 
that they would not get additional resources from local governments given the 
economic climate in their communities. They saw this project as a very important 
State Library initiative and hoped the State Library would continue the initiative as 
it was their “only hope” for staying abreast with new technology and applications. 



User Focus Group 

Six individuals participated in the user focus group. The library director also was 
present. The group included a: 

• Male pastor and writer, 43 years old 
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• Male manager of a used bookstore, 51 years old 

• Female free lance writer, 39 years old 

• Male college student, 24 years old 

• Male former restaurant manager, 55 years old; and 

• Female management consultant, 44 years old. 

Five had college or graduate education and one had a community college degree. None 
considered themselves as “new users” to the library. Four saw themselves as 
“beginners” and two as “intermediate users” in terms of their Internet skills. For five 
of the six, their first ever access to the Internet came through the OnLine at PA 
Libraries project. 



Awareness of the Internet Access 

All of these users heard about the Internet connection as a result of coming to the 
library. They saw the workstation and were interested in using it, talked to the 
director about it, and received basic instruction from him. Then, as one person put it, 
“I was off and running.” They had not received much assistance from library staff 
and “learned by doing.” 



Use of the Internet 



All of the participants were “amazed” and “impressed” with the information 
resources available over the net and more specifically on the Web. Each had one or 
more success stories of how they found information that they otherwise would not 
have been able to obtain. Topics that were of interest to them for which they had 
used the Internet were employment opportunities, information on companies, 
financial investments, music, hobbies, and the Civil War. One told of getting the 
n am e and address of an organization in a foreign country; another about pictures of 
paintings in a particular museum; and yet another about historic needlework with 
pictures of that needlework. 

They noted that the general use of the public access workstation seems to be 
increasing. Two agreed that they now phone ahead to make a reservation to use the 
workstation because they do not want to wait when they arrive at the library. There 
was a general consensus that once the kids knew about the Internet from the schools 
it would be very difficult to gain access to the workstation. They asked the director 
when a second public access terminal would be available. 
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Access for Economic Development 

Three and possibly four of the respondents used the Internet as an entrepreneur. 
One person commented that she uses various websites to “check-out opportunities 
for freelance writing.” She noted that a number of sites on the Web had given her 
ideas or direct opportunities for selling some of her articles. This person referred to 
the public access workstation as her “lifeline” to the world. Another was trying to 
start up a virtual bookstore on the Web. Another talked about the possibility of 
selling crafts via the web. This discussion highlighted the view that users could use 
the web to compete in a global marketplace, “bring commerce” to the county, and 
that they were using the library’s Internet connection to support local economic 
development. 



Additional Resources for the Library’s Internet Connection 

The users were a bit disparaging about the likelihood that the Internet connection 
would result in more local government support for the library. One person stated that 
in his view, “anything new was good for the county,” was not the view held by most 
everyone in the county - especially local government. They stressed that this was a 
very conservative county, that no one would be willing to support tax increases for 
anything - least of all the library, and that most people would likely see the Internet 
connection as a “frill.” They asked the library director how the public access terminal 
would be maintained after the end of the project. 



Conclusion and Recommendations 

This group of users were avid supporters of the library and the public access 
Internet connection. They believed that such innovations at the library were critical 
if the county was to move forward toward the 21st century successfully. Indeed, they 
thought this particular county could really profit by the public access terminal 
because it was so poor, rural, and isolated. They also noted that their views probably 
were not the views of most people in the county. 

One person commented that use of the Internet via the public library was 
especially important because many people in the county could not even have TV 
reception unless they paid for cable - due to the isolation of the county. Thus, her 
view was that in such a situation, having a free Internet connection via the public 
library was absolutely essential. Without it, some county residents mig;ht “never 
learn about the bigger world out there.” She felt the county was very provincial and 
the public access Internet connection would reduce that provincialism. 

This group of users did not want to lose the public access to the Internet and were 
concerned about how the service would be maintained and expanded in the future. 
They saw a need for a second public access terminal now; they liked the idea of having 
more training sessions and thought maybe a local “expert” might volunteer to do this; 
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and they thought additional guides, magazines, or handouts about information on the 
Web woxold also be useful. 



LIBRARY B 

This library has a total operating budget of $95,000, of which $1,100 is budgeted 
for books and $2,500 for serials. It is attractive but small (5,400 square foot), 
relatively new (built in 1974) and located in a municipal complex. There are three full 
time staff at 35 hours per week and two part time staff at 15 hours per week. The 
library director and staff are considerably underpaid -- especially the library director 
who has a MLS. The library is open 50 hours per week. Participants described the 
local area as economically “stagnant.” The school district is the major employer in 
town and agriculture comprises much of the local workforce and the population is an 
“aging one.” 



Interview with Director 

The director was very pleased to have received the grant for participating in the 
OnLine PA project - oftentimes it is difficult to compete against the bigger libraries 
for grants and resources. She noted that all information technology (IT) at the library 
had come from grants or gifts of one kind or another as there was no budget for IT 
available. Her district librarian has been “superb” in helping the library with getting 
them this grant. The director was very aware that the library was “well behind” in IT 
use and she saw this grant as a way to “get up-to-date.” 



Workstation and Connectivity 

The workstation is in a prominent location very close to the main 
reference/circulation desk. They had no difficulties in setting up the computer. 
During January 1997 it was used by staff for training and in early February the 
workstation was made available to the public. A local Internet Service Provider 
donates the connection and provides unlimited use. The library does pay for the 
additional telephone line. 



Training 

Staff had very limited knowledge of the Internet, the PC workstation, and 
telecommunications. The first training session the director attended, sponsored by 
the project, had content that oftentimes was “over her head” with unclear 
terminology. Terms like “dot com” meant nothing to many of the participants. They 
needed basics on how to use the computer before learning how to use Internet 
applications. It probably would have been more helpful if staff could have been 
trained at this time. The second training workshop was much better and more helpful 
but probably should have been scheduled earlier since it dealt with reference 
resources. 
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Staff provide users with basic information only on using the workstation. Only 
one member of the staff is really “into using the workstation and the Internet.” This 
person has trained other staff and also helps the director. She also has set up special 
bookmarks for “important or key” websites. There have been discussions at board 
meetings to have a training session for the public. They are considering a weekend 
and evening session to get all types of users. They are also tr3dng to get community 
volunteers to help with community training. The director believes that they have 
some users who would be very good as trainers. 

The director noted the importance of ongoing training for both staff and users but 
also noted the lack of time, difficulty in motivating staff-- especially the elderly staff- 
to participate in training. What they need most is the time to simply sit at the 
terminal with someone who really knows what they are doing and get “one-on-one 
training and help.” She concluded that “there never will be enough training” done or 
available. 



Users 



There are clear generational issues with computer use: both staff and patrons. 
Many do not see the value to computer use just yet — both software applications and 
Internet. In addition, lifestyle factors seem to affect who uses the Internet 
workstation. Professionals, younger folks, and kids that have access in school seem 
to be “typical” users. A number of the users came to the library and needed little help 
in using the workstation. There appears to be a number of repeat users and a set of 
core or regular users of the Internet workstation. 



Policies and Procedures for Use 

There is a sign-up sheet for users to access the Internet workstation. Users have 
a 30 minute time period to use the workstation. If no one is in line or has signed up, 
then they can stay on longer. There is a written consent form that children under 18 
have to sign with their parents (while parents are present) if they are to use the 
terminal. As of early June the library had 47 such signed forms on-hand. A sign on 
the workstation states “No Chat Rooms.” They have encountered only one “problem 
patron” in the use of the workstation - having to do with chatrooms. There have 
been no problems with pornography. The director believes that having the 
workstation so close to the main desk and “watchful librarians” helps to minimize any 
such problems. 



Increased Visibility and Credibility 

The library is an unknown resource in general. The OnLine at PA Libraries 
project seems to have increased credibility/value of the library. She believes that 
there is increased community awareness of the library as a result of the project. 
There has been some press coverage (one article in the local newspaper earlier and 
one currently being written by the director). The director also noted that increased 
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community awareness and visibility take time. While use of the Internet workstation 
is increasing, the full potential of the terminal still may not be realized by the time 
the project ends. 



Role of the Board 

The library board is a governing board that reports to city council. The library 
board has had some interest but has not been particularly active in the project. The 
board, like the community, is not technologically savvy. The board has a good 
relationship with the school district which might help in working with the schools in 
the future regarding the Internet project. While the board is supportive about the 
terminal and Internet access, they have not made an effort to come in and learn it. 



Too Much to Do 

The director pointed out that learning to use the Internet workstation, helping 
users, integrating the Internet access into other library services is in addition to 
everything else that they are trying to do. Other responsibilities did not “go away’ 
with the arrival of the workstation and Internet access. Indeed, as she noted, were it 
not for dedicated staff (underpaid staff at that) they could not get done what they c^. 
So, the arrival of the Internet workstation needs to be seen in the context of all the 
other responsibilities they are trying to complete — with inadequate staff, time, and 
budget. 



Conclusion 

The director stated that having the equipment or getting the equipment is always 
an issue. But the real issue is knowing how to use it. She recognizes that she and 
the staff really do not know how to take advantage of the new computing technology. 
They need, in her opinion, a champion/super user to really cull the resources on the 
net, set up bookmarks, and help the library staff better use the Internet connection. 

The State Library did an excellent job in starting this project. Bell Atlantic also 
should be congratulated for funding the project. If it were not for this project the 
library would not have an Internet connection nor would they likely have one in the 
near future. Although there were problems with mechanics (training, use, time, etc.) 
this project was well worth the effort. The director stated that follow-up projects that 
build on the success of this project will be needed, however, if the library is to 
continue to use computers and new technology.” 
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Group Interview with Staff 

Four staff members participated in this interview session. One was full-time 
during the day, the other three were part-time and worked primarily in the late 
afternoons or evenings. Two of these staff are paid by a federal program, otherwise 
the library could not afford to hire them. All worked in the hbrary prior to the OnLine 
at PA Libraries project. They had very mixed perceptions and expectations about the 
project: some were frightened and did not want to learn it; others found it a welcome 
tool. One staff member clearly was a “champion” for the Internet connection and 
workstation — others deferred to her views and comments. 



Barriers 



For a number of the staff, simply using a computer and a mouse was a big first 
step. One participant stated, “look, we’re old and set in our ways, it’s very hard to 
change and use these computers.” They all noted that they were “overloaded” with 
work and things that needed to be done. The Internet workstation, although it might 
be neat, also is simply more work. One person indicated that this whole “Internet 
stuff’ was beyond her! 



Success Stories 

The one staff member, especially, is discovering Internet access as an important 
tool. She told of an instance where a class needed information on the environment 
and ever 5 fthing in the library was already checked out. She used the Internet to a 
number of Web sites and was “amazed” at all the information available and in full 
text. The students would never have had access to this information had it not been 
for the Internet connection. She noted that the connection really expands access to 
resources and tools; she would not be able to help some patrons without it. 



Training 

The one staff member really took to the Internet access. She is now getting a 
connection at home as well. This person and one other (not present at the interview) 
provide training to others on the staff. The other staff and the director generally rely 
on these two staff members to answer questions about both Internet and equipment. 
They estimate that they have provided staff with approximately 20 hours of training 
~ most of which they have learned on their own. 

In terms of training users, most users already have some skills. The library policy 
is such that users do not receive much training due to lack of time. However, during 
the day they do answer some basic questions for those that are not familiar. In such 
instances they may spend 5-10 minutes and users are generally off and running. 
Once connected, many users easily figure out how to use the Web. In fact, the staff 
oftentimes learned from the users. 
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Users and Use of the Workstation 

The staff reported that users are very positive about the Internet and are 
amazed at how far they can go with obtaining Internet resources. There are many 
repeat users. They characterized the users as young students and young men and 
some women. Users especially liked printing capabilities so they can immediately 
obtain copies of important information. One stated that the “range of resources I can 
access on the Internet is very impressive.” One staff member has gone out and found 
“important reference resources” and has bookmarked many. She does not really use 
the Mansfield Library homepage but has her collection of favorites. 

When asked for examples of topics they have found to be especially good for 
patrons, a flood of responses came, including; 

• Financial information; 

• Sports; 

• Recreational; 

• Genealogy; 

• Employment (very high use in this area); and 

• E-mail (out only). 

They thought the workstation was generally in use most all the time. There had been 
no problems thus far with pornography, vandalism, etc. 

On the other hand some users think they can do more than they actually can and 
had unrealistic expectations. Some have the view that “one button for everything” 
and were disappointed when it took more effort. Patrons want more terminals and 
faster access - the 28.8 baud modem just is too slow. The schools are just starting 
to have Internet and to teach it. 

The schools accept Internet-based resources as sources for papers. Students are 
starting to expect the library to have the Internet sources since some have this in 
school. Staff expect significantly increased student use in the future once the schools 
are more online. 



Visibility 

Generally, the library relies on word of mouth to publicize the Internet access. 
Notices, etc., would come from the board and director -- the staff is really not involved 
in this and, as a result, they did not know of any publicity to date. They found that 
word of mouth is helping to bring new users/patrons because of the OnLine at PA 
Libraries workstation. 
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Conclusion 



The staff, and especially the one individual who was “into” the Internet believed 
that the OnLine at PA Libraries project was an exceptionally good project and, 
despite extra work, thought it was a very good thing for their community. In terms of 
specific impacts on the library, they thought Internet access had the potential to 
reduce TTf- since a number of items formd in full text would not have been available in 
the library. In addition, they believed that Internet use may be increasing circulation 
since people would find information on topics and want additional sources that they 
could take home with them. They had the following recommendations for future 
activities: 

• Increase the number of computer workstations - it was becoming increasingly 
difficult for the staff to have access to the workstation for reference uses. 

• Obtain faster modems or obtain better connectivity through leased lines. 

• Provide more in-house, hands-on training; the District Library should be more 
involved in training; they could provide in-house “roving” trainers. 

• Improve the ISP connection - it oftentimes “times out” and the next patron 
may not know how to re-connect. 

• Provide more time for staff to be trained and use the Internet. 

One staff member said that she was really pleased that Bell Atlantic funded this 
project - that patrons did notice Bell Atlantic sponsorship (through signs on the 
terminal, the OnLine PA mousepad, and explanations). They liked the State Library 
for doing this (but did not know that the State Library organized it!); everyone was a 
winner on this project. 



Luncheon F ocus Group with Neighbor Librarians 

The luncheon focus group session contained nine librarians and one student intern 
from near-by libraries including the librarian from the site library. One participant 
was from the District Library and another was a staff member, all others were 
directors. One participant had not been involved in the project as yet but had an 
application in for “round two” participation. These individuals represented libraries 
with budgets from $20,000 to $130,000 per year. The focus group session lasted 
approximately an hour and a half and was extremely lively. 



Overall Perceptions of Project 

Generally, everyone had very positive views of the project, its operation and 
administration, and the impacts on their libraries and the community. The telephone 
charges they paid for connectivity were in the range of $15-$29 per month and this 
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really was the only ongoing expense they had to cover. Except for one who had 
previously had text access, they strongly agreed that “if it had not been for this grant 
we would not now have Internet access.” One library had text only access to the 
Internet prior to the project, and the grant provided a significant upgrade in the 
quality of the access provided. 



Leveraging the Grant for Local IT Support 

The participants were very clear that while there were numerous benefits and 
impacts from these grants, they did not result in obtaining additional resources for IT 
and computing from local government for their library. “The reality is that there is no 
extra money in my local government and they think that too much already goes to 
the library.” Local government officials were “pleased” that the library got the grant 
— if they knew at all -- but saw it as an “extra” for the library. 

The participants agreed that the only way they would be able to upgrade this 
technology or build upon it would be with future grants or other types of assistance 
from the state. Thus, the model that providing rural public libraries with Internet 
connection would “hook” local governments to provide additional support for the 
library in general or IT in particular simply is not accurate. Other benefits and 
leveraging of the connection did take place (see below), but not in terms of obtaining 
additional local support (as yet). 



TLse.s and Users 

Participants saw the Internet connection as supplementing and enhancing their 
resources and also recognized that the access provided them with resources that 
otherwise they could never hope to acquire and use. Many had specific stories or 
anecdotes about locating resources on the Internet that were “astounding,” 
“fabulous,” or “unbelievable.” Specific topics of useful resources included: 

• Government documents; 

• Medical and health information; 

• History (anecdote about person interested in the Civil War); 

• Job opportunities; and 

• Genealogy. 

The nature and types of users seemed to vary somewhat from library to library. 
There were divergent views as to whether the Internet connection was used most by 
“repeat users” versus new users; but all agreed that new users had been attracted to 
the library because of the Internet connection. Most agreed that the younger, more 
professional, and kids who knew about the Internet from school were “typical” users. 
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All had printers available at the rate of 50 - 250 per page and patrons took 
advantage of printing information they found on the Web. 



Role of Library Boards 

Here too there were very different views about the interest and role of the boards 
in the project and in Internet use and access. The reality is that many of the board 
members were elderly and simply did not understand the new technology and why it 
was needed in the library. Although they may not have been anxiously supporting 
the Internet access they generally “went along” and thought it was OK - especially 
since it did not cost them any money. The participants doubted if the board members 
could be proponents for additional library IT projects since they understood little 
about it. Most participants agreed that only a very few of the board members had 
come in to use the Internet workstation or see (ask for) a demonstration. 



Problems and Issues 

The participants identified a number of problems and issues related to the project 
that might require additional attention and thought: 

• Original default settings on the workstation and modem . A number of the 
participants commented that the workstation arrived with settings that they 
wanted to change or improve (i.e., baud setting on the modem, system 
configurations, colors available on the monitor etc.) and were told that they 
could not be changed by the lU. Most were quite passionate about not being 
able to obtain the “password” to disable the software program Fortress. 

• State Library’s sunnort for rural public libraries . The group agreed that the 
State Librarian really went “out of his way” to help rural public libraries with 
this project and it was really welcome since some thought the larger urban 
libraries tend to get more attention. This project “really made the State 
Library look good” and participants hope others will come along as well. 

• Rnla of lU 29 in the nroiect . There were some comments (not by all) regarding 
the amount of money that went to the lU to administer the project, provide 
support, training, etc. Also many felt that the administrative structure of the 
project was somewhat rigid and did not allow flexibihty at the local level. 

• Training and support . The only “real” training was at the second training 
session where they learned “what to do” with the Internet connection. 
Information and advice from the lU oftentimes did not help them much. One 
person commented (with others nodding agreement) that they just quit calling 
with questions and found someone locally to help them. In addition, provision of 
some small amount of resources for each library, like $500 each, for software, 
other applications, and to help in travel to the training would have been very 
helpful. 
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• Budgeting and Planning for IT . They need assistance either from the state or 
their district library center on how to budget and plan for future IT, computing, 
and telecommunications costs. In addition, they need to know “what next?” in 
terms of how to integrate the Internet access into overall library services and 
resources. One person commented, “it’s great that we have this [Internet 
workstation] but now what do we do after we learn to use it?” 

• State aid formula . There was some discussion about the complexity of the 
state aid formula and how rural public libraries “competed” effectively with 
other libraries in the state. Some thought it would be good if the formula 
allowed some additional support for the rural public libraries to better support 
IT and upgrade their workstation. A number of participants saw “no way” 
that they could find resources to upgrade their workstation from the grant in 
the future. 

Participants identified these issues and problems in a positive and constructive 
manner as a means for them to continue IT development in their library and to 
maintain the momentum from the grant. 



Benefits and Impacts 

When participants considered the range of benefits and impacts that had, or were 
likely to result from the project, they identified the following: 

• Improved commimication via e-mail with other librarians, the District Library 
and others. 

• Access to an “astounding” number of additional resources that they did not 
have access to at their library. 

• Access to those additional resources that were very current and up-to-date as 
compared to many of the print sources they currently owned. 

• Access to those additional resources was full text and very often free. 

• Increased visibility of the library in the community - for some, the library had 
the only public access Internet connection in the entire community and 
increasingly, folks were becoming aware of this. 

• Increased credibility of the library due to the appearance and use of the 
Internet workstation made the library seem more up-to-date. 

• Productivity of library staff in areas of reference and referral and perhaps later 
in collection development may increase. 

• Saving the cost of some expenditures for resources since they did not update or 
replace print sources if they knew they were easily available on the Internet. 
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• ILL costs may be decreasing since users can oftentimes get full text of 
whatever they want right then, and not want to wait for an ILL later. 

• With purchase of additional CD Rom reference products instead of print 
sources could possibly save money. 

• The presence of the Internet workstations in the libraries leveraged the ability 
of the District Library to write and obtain grants; she reviewed the list of 
grants issued in the last year and estimated that some 12 grants (that 
resulted in approximately $35,740) resulted or were significantly 
strengthened because of the Internet workstations throughout the District. 

• The OnLine at PA Libraries project “jumpstarted” many of these libraries into 
the Internet world of information resources which would not have occurred 
otherwise; librarians are now becoming computer literate and are leading their 
communities in the use of computers and the Internet. 

• One librarian noted that they hired a 20 hr/wk staffperson to operate the 
Internet workstation as a result of the grant (others were very jealous!). 

An interesting theme throughout the discussion of benefits and impacts was the oft- 
stated view that it may be too early to identify all the possible benefits and impacts 
from the project. Participants noted that some libraries have only had public access 
Internet available since February (note that others had it available since November). 
But they were very positive that we are only “just beginning” to see the various 
impacts and benefits from the project. They suspected that a year from now, more 
benefits and impacts than those listed above would result. 



Participants noted that the State Library computed the value of the workstation 
they received at $2,500. The investigators pointed out that if they had had to buy 
that same configuration without the large scale purchase done by the state, the price 
would have been considerably more. Nonetheless, they noted that for that 
investment, they thought they had provided significant time, effort, travel, training, 
etc. Thus, the $2,500 investment resulted in considerable return on investment for 
the state. Bell Atlantic, and their local commrmity. 

These librarians were especially interested in “next steps” with this project. One 
asked: “what is the plan in the future for computing development at my library? does 
the State Library have another grant program planned? will the district library 
center be more involved in assisting us in IT planning?” For some participants, there 
was a sense of “now what?” and what was “Phase II” going to be! 
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Users Focus Group 

The focus group session was, in itself, a dynamic and evolving situation. At the 
outset, five individuals arrived to participate in the session who had been contacted 
earlier. However, a number of Internet users were present in the library at the time 
of the focus group and were “ushered” into the session by library staff. In addition, 
half way through the session a man passing through the city used the Internet 
workstation and joined the session. In addition, some of the teenagers that joined the 
session after about 10 minutes into the session left after another 10-15 minutes. The 
session lasted about an hour. 

Overall, there were 8 participants who were fully engaged in the focus group for 
the majority of time the session was held. This included five adults and three 
teenagers. Adults included a teacher, a teacher’s aid, someone in automotive sales, 
and an inventory analyst. The adults were between 30-50 in age. The adults 
characterized themselves as beginners, used the Internet connection 1-8 times during 
the past month, and had no formal training in the use of the Internet or the 
workstation. They ranged from having received high school education to graduate 
education. The teenagers reported using the connection between 10-15 times in the 
last month and saw themselves as “intermediate” users. They also had received no 
formal training. 



Users and Uses 



The participants reported a variety of uses of the Internet. They use it for a range 
of reasons: doing homework, book reports and games for the teenagers; recreational 
uses such as tourism for adults; checking on competition for business purposes 
(inventory and sales of car parts); CNN and other news sources; job information and 
employment; and genealogy - to name a few. The participants thought that there 
was a generation gap of who used the Internet workstation - younger people more 
hkely to use it. 

One teenager commented that it was “faster and easier to get information off the 
Internet than it was to find it in the library.” He did not use print sources much 
anymore. A woman said that she notes the I^L from television advertisements and 
finds out sites that way. A parent of one of the teenagers commented that she 
simply could not believe the amoimt of information available and was “amazed.” In 
addition, she relied on her son to help her find “stuff” when she needed it. Another 
teenager commented that “my Mom has to pull me away [from the terminal] when 
we have to leave.” Each had some success stories about finding information on the 
Internet that they otherwise believe they would never have foimd. 



Pornogranhv 

The focus group participants indicated that they had not been aware of any 
problems with users trying to obtain pornographic information or using the 
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connection for inappropriate uses. A teenager said that with the terminal so close to 
the librarian’s desk and closely monitored by the librarians, it would difficult to “get 
away with watching pornography anyway.” There was also the sense from the 
teenagers that they knew they had signed an agreement to not access pornographic 
material and knew that the consequences of being denied use of the connection were 
significant. 



Connectivity Issues 

Four of the participants had connections to the Internet at home. One stated that 
she came to the library to use the Internet because the phone line at home “was 
always tied up.” One stated that she had very poor phone service where she lived and 
could not use a modem because she had to tell the operator her ID number in order to 
get a connection. This person indicated that she would pay for being able to use the 
Internet since she could not access it from home. 



Trsdriing 

None of the participants had any formal training in how to use the Internet or the 
workstation except, some noted, some introductory information from the librarians. 
One teenager proudly stated that he knew “a lot more” about the Internet than the 
librarians did and that he often helped them find information. One of the adults said 
that she relied on her son for help, another learned by “trial and error,” and still 
another said he “learned by doing it.” The local schools either are just starting or do 
not provide access to students. 

One of the participants was a member of the library board and asked the teenager 
who was very good at the Internet if he would be willing to help in giving training 
sessions to the public if such could be organized. He indicated that he would and there 
seemed to be some impetus to develop training sessions for the public offered by the 
library and volunteers in the future. 



Recommendations/Conclusions 



They thought that additional terminals were needed both for the pubhc and for the 
librarians. This would be especially true now that Summer has come and school is 
out. In addition, as the schools begin providing Internet access and training, that will 
drive additional use by students. A potential “invasion” by students seemed to 
concern one of the adults as to her ability to access the Internet at the library. They 
thought that the cost of paper was not problematic (after 3 pages, 5<Z per page). One 
teenager noted that the speed of connection could be bad and wanted to know if the 
library could obtain a faster connection. 

This group of users was especially interesting in that it included a very wide range 
of individuals including a number of teenagers. One teenager, age 15, was especially 
knowledgeable and interested in the Internet access from the library. He seemed to 
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have been responsible for training other teenagers and helping library staff as well. 
Overall, however, the users were just getting started with the Internet. They believed 
that the Internet connectivity was a “great thing” for the library, they are “amazed” 
at the soruces that can be foimd and obtained, and have told others in the community 
about the new service. 



LIBRARY C 

The library’s budget is $236,000 with approximately $58,000 for materials. There 
is the main library as well as two branches. The library is located in a county with a 
population of 32,000 and with a very high growth rate. The growth rate is attributed 
to families moving to the county from New York and New Jersey to “avoid taxes and 
live in a more rural setting.” The facility is an old historic landmark building in the 
town with very small cut-up rooms and cramped quarters with only 2,100 usable 
square feet. There are nine staff of which three are full-time. The director reports to 
three county commissioners and a board that is advisory in nature. Despite the 
growth in population, some thought the area was economically depressed with the 
school district being the largest employer. 



Interview with Director 

The director is relatively new to this position although she has worked in 
Pennsylvania libraries a mrniber of years. She was very supportive of the Online at 
PA Libraries project from the beginning, believing that public access Internet 
services would be especially good and useful in this county. They received the 
workstations and equipment and connected to the Internet in November, 1996. She 
participated in both training sessions provided by the lU, thinks the project has been 
extremely successful, and is appreciative to the State Library and Bell Atlantic for 
supporting the project. 



Context of the County 

The county is experiencing rapid growth without much economic development. 
There are two key groups of individuals in the county. The first group might best be 
referred to as the natives - folks who have lived in the county most of their lives. 
They may own large amounts of land, some of which is being developed for resorts or 
vacation homes. The second group are recent arrivals who are fleeing high taxes from 
New York or New Jersey and are willing to do an horn- and a half commute (each way) 
to the NYC-NJ area from the county. They may also have second homes in the 
county but live in NYC-NJ area. Thus, both groups do not want tax increases and 
are very conservative about how money is spent and for what in the county. The 
growth, however, has an effect on the county infrastructm-e and need for services. 
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Connectivity 

The library pays $40 per month for the phone line. After some initial problems 
with the ISP, a new one has been very good and provided reliable service. The ISP 
donates the service to the library and also has provided some assistance on general 
maintenance of the connection and the computer. One of the employees of the ISP 
was a heavy library Internet user who xmtil recently knew nothing about the Internet 
and learned about the Internet from the library (see users section below). As an 
employee now at the local ISP he helps the library out from time to time with a range 
of assistance. 

The library has a second Internet connection on a 14.4 dial-up on a 486 
workstation not available to the public. This workstation is “the workhorse” for 
administrative and other office support. There also is an automated circulation and 
catalog system which is networked with five terminals. 



Support from County Commissioners 

She noted that the county commissioners, while supportive of the OnLine at PA 
Libraries project, were not actively involved in it — although one has come by the 
library to “try it out.” Thus far, the project has not directly resulted in additional 
funds or support for the library from local government. While she is certain that the 
commissioners are pleased about the library’s participation in the project, the 
message of “get more grants” is likely to be the response from participating in the 
project rather than let us give you more money for IT at the library. 



Training 

The director attended both the lU training sessions and thought that both were 
excellent and very helpful. Upon return from the first, she provided a training session 
for her staff and sat with each providing a hands on, one-on-one session as to how to 
use the new workstation. She estimated that about half the staff had “taken to” the 
workstation and using the Internet. She noted that she did not think “forcing” staff to 
use the workstation was the best approach, but instead facilitated and supported 
their learning efforts. There was one staffimember who really has gotten involved in 
the Internet and that person has helped the director mount training sessions for the 
pubhc. 

Once a month the director and a staffimember provide a training session for the 
public. These have been well-attended. The director tells those who attend the 
training session that she expects each attendee to provide at least 2 hours of 
volxmteer help to others in the library on how to use the workstation and the Internet. 
She believes that this approach has been very successful. Most of the attendees at 
the training have given back more than two hours of volunteer help to the library. 
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Importance of Volunteers 

The director plainly stated that “without the involvement of the volunteers, we 
never could have been as successful with the OnLine PA project.” She could not 
provide enough praise for the volunteers and their help with training, working with 
users, and being involved in the project. In a session with two of these volunteers (see 
below) their interest and involvement in the project was obvious. A number of the 
volunteers regularly provide 10-12 hours per week assisting the library, many others 
provide 2-3 hours per week. 

The director made a very interesting point that a number of these volunteers 
volunteered less to help the library and more because they were interested in the 
Internet, computing, etc. Thus, to some degree the training program for the public 
and the resulting volunteers produced new users for the library — users based 
primarily on their interest in the Internet. Many of these volunteers had 
sophisticated knowledge and interest in the Internet. The director commented that 
“these volunteers have saved us both with training and in some maintenance of our 
workstation and connection.” 



Pornography and Use Policy 

The library currently has a use policy which they ask everyone to read and sign 
prior to their use of the public access Internet workstation. This policy statement 
covers a range of topics and makes the policy for use of the workstation quite clear. 
Users under 16 have to have their parents sign the policy; those 14 and 15 may be 
exempted from the parent being with the child while using the Internet - otherwise, 
the adult is expected to be present while the child uses the Internet. 

Although the workstation is a bit hidden from the main desk, there have not been 
any incidents regarding pornography or inappropriate use of the Internet connection. 
The workstation, however, is very public and it is easy to see the screen as one 
passes by the area. The director reported that adults had not complained about 
having to sign the policy statement prior to using the Internet connection. 



Marketing 

The director provided the study team with a number of stories that appeared in 
the local paper, announcements, etc. related to the Internet connection. These had 
occurred about the new service, about training, need for volunteers, etc. The director 
had some concern about doing much more marketing as the terminal use had rapidly 
increased. There were many people using the connection, in fact, it was becoming 
increasingly difficult to use it without making a reservation in advance. 
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Work Load 

The director noted that the workstation and Internet connection came with no 
additional staff to help operate them. Thus, she had successfully (“Thank God”) been 
able to rely on volunteers for assistance. But the reality is that overall the project 
resulted in an increased work load for the library and primarily for her. She was not 
complaining or whining about this — just stating it as a fact. She also noted that she 
is paid for 35 hours of work per week and t3^ically provides 50 hours and sometimes 
more like 55. She is concerned how the library will transition successfully into this 
new information and computer age without additional staff and staff who have 
computing and networking skills. 



Benefits 



Without a doubt, the director stated, the arrival of the workstation and Internet 
connectivity has been a major boon for the library and its community. The library 
has received significantly increased visibility in the county and new users have come 
to the library “specifically to use the Internet.” She also was convinced that the 
connection gave the library a sense of being “with it” in terms of having current 
technology — thus, adding to the library’s credibility as “key player” in the information 
age. She thinks that this new image of the library has helped it obtain some 
additional donations and gifts. 

She also pointed out that for a small library such as hers, the access to an 
unbelievable amount of information and reference resoxirces was phenomenal. She 
related numerous success stories of finding information or assisting users with the 
Internet connection that never could have occurred were it not for the Internet 
connection and workstation. 



Issues and Recommendations 

The director would like to have another terminal that is primarily for staff use 
since it is very difficult for staff to use the public access terminal for training and 
reference use — given its heavy use by the public. Indeed, the main desk is in a 
different room from where they had to place the OnLine PA workstation. Although it 
is clearly a project objective to provide “public access” to the workstation, she wants 
to be better able to use the Internet for reference assistance. 

This library would not have been able to purchase a workstation and provide 
Internet connectivity without the grant from the OnLine at PA Libraries project. 
Nonetheless, the library has raised some expectations for library services in the 
county that the library may not be able to meet in the future. It is unclear how the 
equipment can be upgraded in the future or how additional information technology can 
be obtained at all for the library. The bottom line, however, is “it is a very good thing 
that the State Library and Bell Atlantic were able to do this grant.” 
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Meeting with Volunteers 

Originally, this meeting was to include two staff members and two volunteers who 
have been actively involved with the OnLine at PA Libraries project. The two staff 
members, however, were unable to attend the session. Thus, the interview occurred 
with the two volunteers and in the presence of the director. Both volunteers were 
retired, one a man, and one a woman. Both had professional level experience in 
previous work, and both were extremely active and interested in the Internet 
connection at the library. 



Types of Users and Uses 

Both volunteers agreed that it was very difficult to describe the “t3^ical” user of 
the public access Internet connection. One commented that he was surprised at the 
number of elderly users at the workstation. The other commented that the users 
“come from all walks of life.” 

Each had a number of success stories about helping people find information that 
could not have been obtained except via the Internet “and certainly would not have 
been available here at the library.” The t3Tpes of topics they mentioned that were 
heavily used included; 

• Financial information; 

• Health and medical information; 

• Jobs and emplo5onent opportimities; 

• Individual research questions; 

• Government information; 

• Genealogy; and 

• Student papers/assignments. 

Again, they noted that they were surprised by the range of topics and subjects that 
people used the Internet access to obtain. 



Library Internet as Social Equalizer 

One volunteer commented on how impressed she was that access to the Internet 
and being able to search it for information was “a great equalizer” among people in 
different classes of society. She pointed out that public access to the Internet gave 
“anyone who wanted it” access to the world of information - regardless of race, social 
class, etc. This, she said was an important role for the library. The other volunteer 
agreed emphatically. They both thought that this role of the library was especially 
important given the make-up of residents in the county. 
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Rpinpr a Volunteer 

Both were very pleased to have been serving as an Internet volunteer. Both had 
attended the public training provided by the director and one said he had been 
“hooked” to help out at the library. He estimated his weekly involvement with 
volunteering at the library to be about 12 hours per week. This particular volunteer 
was extremely knowledgeable about the Internet and very engaged in being a 
volunteer. He told of instances when he would see patrons “stumbling” to find the 
information they needed and directed them to the Internet connection, helping them 
to find what they wanted “almost immediately.” One could see how his presence in 
the library would be a positive dynamic supporting the use of the connection. 

They both felt that it was somewhat of an “honor” to be an Internet volunteer at 
the library. They both provided quite a bit of “word-of-mouth” advertising about the 
Internet connection. They had a great deal of praise for the director in supporting the 
project and the volimteer effort. 



Benefits 



The volunteers noted that there was no institution for higher education in their 
county and without the Internet connection at the public library one would have to 
travel some distance to an academic institution to be able to have such access. In 
fact, one reported that a user commented that with the Internet connection at the 
public library she could “escape from this county.” Both thought that brining in the 
global wealth of arts, information, history, etc. to their county was a significant 
improvement on the quality of life, -t 

They also agreed with the director that the connection had significantly increased 
the visibility of the library. The library now had a “technological presence” in the 
county. They recognized that the range of information they could get, in full text, and 
“right now” was unbelievable and provided information that the library could never, 
otherwise, obtain. 



Recommendations and Issues 



The volunteers said that the library needed two more terminals and connections: 
one for the public and another for the reference staff. They also would participate in 
additional training efforts if available. As an example, one of the volunteers noted 
that if they had been told about how to download Adobe Acrobat sooner, they could 
have been presenting information in pdf format (actual print page presentation). 

They would like to promote the service more, but as with the director, are 
concerned that the library may not be able to meet the additional demand resulting 
from such promotion. They both agreed that the library needs “much better” 
connectivity and that reliance on a 28.8 baud modem was “clearly inadequate.” One 
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was especially interested to know what the “follow-on” project would be that would 
btoild on the success of this effort. 



Luncheon Focus Group with Neighbor Librarians 

Lunch included a total of nine librarians knowledgeable about the project including 
the director of the site library and two librarians from near-by district offices. Those 
managing libraries had budgets ranging from $101,000 - $360,000 with populations 
served ranging between 5,400 to 50,000. Of the nine responses by the participants 
regarding their assessment of the overall success of the project (l=very unsuccessful, 
5=very successful), eight responded with a 5 and one responded with a 4. On average 
the participants have 11 years experience in their current library position. Most had 
experience with computers prior to the project. This was a group of very committed, 
dedicated, and involved librarians. 



Users and Uses 



As a group they estimated that 20-25% of the users to the Internet connection 
were new to the library and came because of the availability of the public access to 
the Internet. They all rely on a sign-up sheet to control use of the workstation, 
although some vary in amount of time for which users can sign-up. They found it 
hard to describe a “typical” user and pointed out that students, seniors, young 
professionals, and others were regular users of the workstation. Most said that the 
workstation was “occupied” most of the time the library was open. 

Their users generally “could not believe” the range of resources and information 
that they could access via the Internet. Users were especially delighted to be able to 
print out full text of interesting articles and other sources. In two cases, local 
teachers assigned students to obtain sources for papers from the Internet connection 
at the library — which also increased use. High use topics that the librarians saw 
users accessing regularly were: 

• Financial information; 

• Genealogy information; 

• Real estate and relocation; 

• Drug prevention information (especially among teenagers); 

• Medical and health information; 

• Political information/current affairs; and 

• Foreign source information such as international newspapers. 
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They agreed that it was difficult to single out specific topics since the range of 
information obtained through the Internet was extensive. 



Control over Workstation Access 



In terms of keeping track of who the users were, one librarian told that at her 
library, first you had to be a registered borrower with a library card to use the 
workstation. Then, to use the workstation, the librarian would “swipe” the card 
through a circulation barcode indicating that the “Internet” was now checked out. 
The desk retained the library card xmtil the individual was done using the workstation. 
She noted that this approach made it very clear who was using the workstation, how 
long, and contributed, she thought, to a lack of pranks or problems developing in the 
hardware and software. The approach also provides excellent record keeping of who 
was using the public access terminal since it was hnked to their library card. 



Gaining Additional Support for the Library 

All the librarians agreed that having the workstation and the Internet 
connectivity while “very good and beneficial” would not pry loose additional resources 
for the library from local government. Each thought their local government was less 
likely than the others to provide additional financial support given local sentiments 
against any type of tax increases. Three of the participants noted that the 
workstations may have helped them in their local fund raising efforts or otherwise 
helped in obtaining some gifts and contributions. 



Success Stories 



Participants each had various success stories from users who accessed the 
Internet tlmough their library. One told of a user who, when quickly finding “exactl3^’ 
what it was she wanted, shouted “Ta Dah,” as if this were the normal procedure. 
Another told of finding information about the effect of coffee on people with Lime 
Disease. Still another told of obtaining some government statistics related to 
economic development in another state that “I never could have gotten — or even 
known about — from the library.” Finally, one told of how they had the Web sites 
bookmarked for Federal and three different state sites that provided tax forms — “this 
really saved us a lot of time and effort.” While there also were instances when patrons 
did not find what they wanted, it appears that there were many more instances of 
success. 



Use of Volxmteers 



Some participants had developed a volunteer program to assist the library in 
training and use of the public access Internet service, others had not. There were 
divided opinions on the usefulness of such an approach. As stated earlier, the director 
at the site visit was very positive about her experience using volunteers. Other 
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librarians related experiences that were not as positive. The group generally agreed 
that it is hard to generalize about volunteer programs as they were very situation 
dependent, and dependent on the personality of the librarians and the volunteers. 



Benefits 



The librarians discussed a number of benefits that resulted from the Internet 
connection in their hbrary, including: 

• Expanded collections . Librarians reported being able to access resources that 
significantly expanded the size, quality, and extent of the collections that they 
had in the library. 

• Savings bv not purchasing materials . Librarians noted a number of 
government resources available via the Internet that they would no longer 
purchase; another noted that she would not renew her Books in Print since she 
was able to get most the information she needed from Amazon.com (web-based 
bookstore). 

• Visibility, credibility, and respect . The librarians were especially pleased with 
the increased visibility and credibility that came to the library as a result of 
being in the technology “forefront” of their community. One person commented 
that it was more than just visibility and credibility ~ “the library has increased 
respect in the community and is seen as a very important institution, more so 
than before the workstation.” 

• Onmrrmnitv equalizer . The Internet connection makes it possible for anyone to 
have access to an expanse of information; it benefits everyone. 

They also commented that a recent announcement from Microsoft that Bill Gates 
would provide $400 million in support to public libraries over the next five years has 
had a positive impact on the library’s visibility in the community - especially since 
these libraries already had some level of Internet access. Overall, they noted that the 
workstations provided their community with links to the rest of the world that simply 
would not have occurred without the grants. 



Issues 



While the introduction of the public access workstations clearly had numerous 
benefits, it also reused a number of issues that participants identified: 

• Workload . The librarians agreed that overall, they now have more 
responsibilities and workload because of the workstations. They believed that 
there was nothing they could “stop doing” and that (except in one instance) 
they received no additional staff to help out on the grant. 



McClure and Bertot 



October 1997 



72 



Evaluation of the OnLine at PA Libraries Project 



• Workstation and telecommunication upgrades . They wondered how they would 
be able to obtain upgrades to the workstations they now had or otherwise 
obtain financial support for additional IT expenses related to the Internet. 

• Raising unrealistic expectations . One person commented that the 
workstations and public access to the Internet had raised the community’s 
expectations of what the library could and should be able to provide 
electronically. Additional service demands were being placed on the library for 
which they had no resources to meet. Others nodded in agreement. 

• Access to the workstations bv library staff . One public access terminal clearly 
is not enough in the library. If staff are to use the Internet effectively for 
reference services, a second terminal at the reference desk is needed. 

• Technical support . They need to have someone available to help them with 
technical support, maintenance, etc. Most readily admit that they do not have 
the needed skills, nor can they afford to hire someone who would have the skills. 
Thus, they are dependent on volunteers or the goodwill of their ISP. 



Recommendations 

This group of librarians had a number of ideas and recommendations based on 
their experience with the project thus far: 

• Financing public library information technology-based services . One individual 
noted that the public library commimity in Pennsylvania, as a group, and with 
the assistance of the state library, needed to rethink financing issues related to 
public libraries in general and IT more specifically. The group referenced a 
series of articles in a Philadelphia paper that discussed this concern (“Libraries 
in Distress,” Philadelphia Inquirer, June 1 - June 4, 1997). 

• Need for better connections . Everyone agreed that 28.8 baud modems were 
“the pits” and simply took too long to download information off the web. One 
person had a 56K line into the library and said that even that was inadequate. 
What we need, he said were T1 connections and “we need it now!” 

• Ideas for the “next” project . There was no lack of ideas for what the State 
Library might do as a follow-up project. Some of the suggestions were: 

Technology maintenance circuit rider. Have one person per district that had 
the responsibility of going out and helping member libraries with technology 
maintenance, repairs, and general troubleshooting. 

- Loaner laptops and projectors . Each district should have a loaner for a 
laptop and a projector so that the librarians could do demonstrations to 
appropriate organizations and groups. 
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- Information Technology planning. The librarians need assistance on how to 
develop an overall IT plan for their libraries, a better understanding of the 
library IT components and configurations that might be possible, and how 
to help finance the implementation of the plan. 

Training. They offered a number of topics that would be appropriate for 
training sessions in the area of Web-based reference services, building and 
maintaining a library homepage; how to train users; developing a library IT 
plan; and providing web based services (i.e., interlibrary loan requests from 
home). 

- Additional terminals. Everyone agreed that a follow-up program that 
provided additional workstations with Internet connectivity would be a good 
idea. They agreed that minimally a library needed three workstations, two 
for the public and one dedicated for staff use. 

- Faster Printers . The demand on the printers at the workstations is 
significant and it takes “forever” to print out long documents. “Can we 
have a program to upgrade and enhance the printers?” 

Tutorials for users . The development of basic or introductory tutorials 
either online or on video that all public libraries in the state could use to 
train users would be very helpful and save a significant amount of time for 
the librarians. 

• Better sharing of project information . They thought that much could be 
learned in the sense of “best practices” from others who had participated in the 
project, but that there really was no mechanism in place to share such 
information, successes, ideas, and recommendations. 

This group of librarians was eager to extend the initial project and hoped that some 
type of formal follow-up project would be developed and funded. 



Interviews with Users 

The interviewer had an opportunity to meet with two users of the public access 
workstation. A number of users had previously agreed to meet with the member of 
the study team during the afternoon but only one arrived for the meeting. Earlier in 
the day, during the meeting with the volunteers, a “heav/’ user of the public access 
workstation stopped and chatted with the interviewer for about 15 minutes. These 
two interviews constitute the interviews with users at this site visit. 



User No. 1 



This user was a young man who had done construction work in the past but had 
been injured and could no longer work construction. He became very interested in the 
public access workstation once it arrived, participated in a training session, and spent 
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“many hours” learning to use the Internet and helping others in the library. This 
experience spurred his interest into telecommunications and computers and he 
taught himself related skills — becoming quite proficient. 

In the process of helping out as a volunteer at the library, he applied for a position 
at a local ISP and was hired; the library director wrote him a letter of 
recommendation. He commented to the interviewer that he had more than a job, “I 
have a career now”— because of his introduction to the public access workstation at 
the library. He went from knowing nothing about the Internet to having a job with 
the local ISP between February to June, 1997 — for which he thanks the library. He 
continues to assist the library with some technical support — in fact, during the 
interview he provided some guidance to the director about a technical problem the 
hbrary currently had. 

He was very enthusiastic about the public access workstation at the library and 
pointed out that he was “a success story^’ as a result of the Bell Atlantic grant. When 
he first began to use the Internet access he was as much entranced with the range of 
sources and information that could be found as well as “how it all worked.” He noted 
that the library needed additional workstations that one terminal was not enough; 
that it needed a better connection than the 28.8 baud modem; and that the presence 
of the Internet workstation was a “very positive thing” for the community. 



User No. 2 



This user was a retired woman who had attended a training session and described 
herself as “just getting to know my way around” on the Internet. She came to the 
library one day, saw a sign advertising the availability of the Internet service, and 
went to the training session. She thinks the service is “wonderful,” and believes it 
should be expanded into the branch libraries (the branch closest to her does not as yet 
have Internet access). She lives in a rural part of the county, some distance away, 
and could not afford a computer herself. So, were it not for the public access terminal 
in the library, she would never be able to use the Internet. 

When asked about her use of the public access workstation she replied with a 
broad range of topics: 

• Information about the Council of Trent (found something immediately) 

• Financial and stock quote information 

• Gardening and “outdoor” information 

• Personal research 

• Civic information. 

She talked about the importance of the Internet connection in conjunction with a 
local issue related to computers in the schools. She had been able to obtain 
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information from various web sites that informed her as to the need for and use of 
computers in schools which she intended to use when she participated in the 
discussion about the issue. She said it was “great” to be able to obtain additional 
views and information to bring to bear on this particular local issue. 

While the service was great, she was frustrated by her inability to use e-mail via 
the workstation and other interactive web-based services. She pointed out that a 
number of services on the Internet would provide you with information or updates, 
etc. if you subscribed to the service. In addition, she would sometimes get the e-mail 
address of someone to contact but could not. She recommended that users be able to 
have e-mail addresses so they could “take full advantage of the Internet connection.” 

In one sense she felt proud of the library for providing this service. She believed 
that the library’s involvement in this project was the perfect role for the library: 
educating the community, providing training for new computer-based services, and 
providing Internet access to everyone who wanted it. The seiwice, in her opinion, had 
greatly increased the visibihty of the library throughout the coimty, more people were 
using the library, and the county now had access to global information and 
perspectives that it never had before the project. 

She was surprised to learn that the project was “soft money” and that there were 
no resources for upgrading and continuing the project. She thought that such a 
project was a very important thing for the State Library to be doing and wondered 
why future resources would not be dedicated to continuing the project since, in the 
scheme of Pennsylvania’s budget, this was a “drop in the bucket.” On the other hand, 
she doubted that residents of the coimty would provide additional tax support for the 
library as a result of this project. Nonetheless, she thought the project was a “great 
success,” hoped it would be continued and expanded, and recommended that the 
library needed more workstations, at least one more for the public and one dedicated 
to staff use. 



LIBRARY D 

While this library is classified as rural in terms of population density, it is actually 
suburban in nature due to its proximity to the state capital. The county population is 
204,000, and continues to grow. Growth is fueled by the county’s proximity to a 
major urban area, a power utility. Sprint (although Sprint is currently downsizing), a 
law school, and a liberal arts college which is directly across the street from the 
library. With such employment opportunities, the county has the lowest 
unemplo 5 unent in the state. 

The population served by the library is approximately 45,000, although the 
director stated that “we serve the entire county with our federated library system” 
(more on that below). The library budget for FY1997 is $450,824, of which over half 
comes from state and local funds. Indeed, the library received $192,913 of its budget 
from a county library tax -- only nine of the 67 counties in the state, including this 
one, have such a tax. 
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Of further interest concerning this library is that it is part of a seven-member 
federated library system. The seven member libraries within the county are 
independent of one another in terms of funding and management. The libraries, 
however, coordinate numerous services and are in the process of installing a common 
on-line public access catalog (OPAC). 



Interview with the Director 

The director had been in her position for nine years, and had witnessed a great deal 
of change during that time - from installing the library’s first OPAC to installing a 
new OPAC within all seven federated libraries to gaining access to the Internet. 
Interestingly, the OnLine PA terminal was not the library’s first connection to the 
Internet. The local newspaper, which now is also an Internet service provider (ISP), 
donated a computer and dial-up Internet connection to the library back in October 
1995. With the arrival of the OnLine PA terminal in October 1996, the newspaper 
removed its donated computer but continues to provide an Internet connection fi'ee of 
charge to the library. 

Overall, the director is “very happy” with the OnLine PA project, and considers 
access to the Internet a “wonderful tool.” While the director did not participate in the 
OnLine PA training sessions, she made sure that appropriate library staff did. 



Context of the County 

The county is experiencing both growth and high employment. It is unlike many 
counties in the state as it has sustainable industry in the area, and also serves as a 
suburb to the state capital. Thus, the county is not currently experiencing any 
financial difficulties. 

The county is also one of only nine in the state that has a separate property- 
based library tax. This tax provides the library with nearly 43% of its budget, serving 
as a significant source of funding for the library. The funding is distributed to each of 
the seven federated library system members using a population served- and hours of 
operation-dependent formula. 

The county is in the process of substantial automation projects. Indeed, the 
county has two full-time technology staff — a network administrator and a computer 
technician — that also serve the library system. The computer technician splits his 
time with the county and the seven libraries of the federated library system. The 
network administrator is configuring and maintaining the OPAC for the library 
system, and is currently training library system, staff on the use of the OPAC. 

Although the consultants were unable to meet with county officials, it appears 
that there are significant efforts to automate the county that include strategic 
information technology planning that is conducted through an automation planning 
committee. This committee is headed by the County Automation Coordinator, who 
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reports directly to the county commissioners. The director considered the automation 
process within the county to be open with input from the library welcome. 

This emphasis on technology and automation by the 3 county commissioners has 
resulted in some direct and tangible benefits to the library/library system. For 
example, the library system was able to persuade the county to provide an additional 
$120,000 for the library system OPAC upgrade by tying into the county’s 
automation efforts -- particularly county records (e.g., deed records). The library, 
through the new OPAC, could serve as a public access point to county documents. 

The county population seems supportive of the library, as witnessed by the large 
number of computer (old) donations to the library. Many of the donations were 
recycled throughout the federated library system as OPAC terminals. Indeed, the 
donations served to significantly increase the number of OPAC terminals available to 
the public. 



The Federated System 

The federated system consists of seven libraries, each with its own board. The 
system reports to the 3-member county commission. 

The library system had a collaborative process in place for book loans and 
returns. The new OPAC, however, solidified the cooperation between the system 
members, as the members had to agree to centralized patron and loan records, as 
well as a plan to implement the new OPAC in all hbraries. This included assisting two 
library members to automate, as they were not automated prior to the new OPAC. 
The OPAC is to be installed over the next several months throughout the federated 
system. 

Funding for the OPAC (approximately $800,000) comes from LSCA/LSTA money, 
county funding, and each library member (in proportion to their budgets, size, etc.). 
LSCA/LSTA funds provided $200,000 for the OPAC. 



Connectivity 

The library receives its Internet connection free of charge from the local 
newspaper through the newspaper’s ISP component -- “Cumberlink.” The newspaper 
has even offered to provide the library with additional free accounts should the library 
desire to connect additional computers. Reliability of the ISP has not been a factor at 
this library. The library pays $320 per year for the phone line (about $27/month). 

The director envisioned a time in the near future when patrons would be able to 
access the Internet through the new OPAC. Such discussions are currently on hold 
due to “getting things up and nmning,’’ but are clearly being considered. 
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Training 

Although the director did not attend the training sessions herself, members of her 
staff did. The individuals who did go were both engaged and willing to use the Internet 
in library activities. The consultants were able to speak to one of those individuals 
who commented that the training was generally helpful, but limited in scope and too 
far apart (more on this below). 

Those staff who did attend the training continued to provide some tredning to staff 
who did not attend. One staff member in particular seems to provide the rest of the 
library staff with technical and Internet-based assistance. 

The director noted that there is no on-going training program in the library. 
However, the director is working to develop a small treiining room in the library that 
would provide Internet training to both staff and the public. There is currently a 
temporary training facility in the library for training various member library staff on 
the use of the new OPAC. The computers, however, will be distributed to the library 
system members for use as OPAC terminals. 

The director commented that training staff to update their technical and Internet 
skills was critical to the success of the library’s Internet use. The director remarked 
about the “frustration” that she and her staff felt over the public’s, particularly 
students, higher degree of technical and Internet skills. The disparity between the 
skill sets of library staff and the public created a serious problem -- both practical and 
psychological — for the library: librarians wanted to help patrons with the Internet, 
but in many cases had only limited knowledge themselves. Moreover, librarians found 
it incredibly difficult to gain the expertise necessary as the workstation was 
constantly in use. 



Use and Acceptable Use Policies 

Prior to installing the OnLine PA workstation, the system member libraries 
agreed to a one hour use limit. The libraries are in the process of changing the use 
limit to a half hour. Patrons must register to use the workstation, but do not need to 
possess library cards. 

The system members also developed an acceptable use policy. This policy is 
currently being revised to more clearly hmit the libraries’ liability with regards to use 
of the workstations by minors. The member libraries do not require parent 
signatures or supervision for children to use the workstation. 

Interestingly, this library uses filtering software. However, as discussed in the 
user section, savvy users (particularly underaged) know how to circumvent the 
software. The director intends to use the new policy under development, as well as 
the recent Supreme Court order striking down the Communications Decency Act, to 
remove the filtering software. Thus far, only one user has presented any problems 
with regards to accessing pornographic material. 
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Marketing and Expansion 

The library received a myriad of press and word-of-mouth concerning the OnLine 
PA terminal. The director is, however, reluctant to pursue additional publicity as the 
workstation is in constant use. Without additional workstations for both patrons and 
staff, the director is not actively seeking “more press.” 



With a college directly across the street from the library, the director expressed 
her desire to forge a cooperative agreement with the college for Internet connectivity. 
The college has better connectivity, and it would not be difficult to tie-in to its 
backbone. With a more robust Internet connection/service, the director would more 
actively promote the librarys connection. 



Work Load 



The director commented that the Internet workstation required additional effort, 
training, and service provision by the library’s staff — without sacrificing 
current/regular library services. Indeed the librarian who happened to be working the 
reference desk also assisted users with the workstation to the extent that they could. 
Too often, however, the librarian could provide only momentary help to individuals 
using the workstation as a line would form at the reference desk or the phone would 
ring. The director saw no immediate solution to this problem, since it was unlikely 
that the library would be able to hire additional staff. 

A related, but key issue, is staff value. At present, staff starting salaries are 
$5/hour with a maximum of $6.5Q/hour. With such low salaries, the director felt it 
difficult to motivate staff to jump onto the Internet (although many were interested 
on their own). 

The director was in the process of seeking volunteers with technical experience to 
assist the library provide Internet-based services. While the library has access to 
some technical assistance from the county, the director would like to see more direct 
Internet-related assistance — e.g., PC configuration, searching and browsing, 
reference, etc. 



Benefits 



The director considered the benefits of the OnLine at PA Libraries project to the 
library to be numerous: 

• It “put libraries on the map” within the community. Libraries had always been 
viewed as the “step sisters” of network-based information, and the OnLine at 
PA Libraries project provided libraries with an opportimity to come to the fore. 
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• It provided an immediate and noticeable expansion of library resources. The 
library was now able to gain access to a number of previously unavailable 
resources. 

• The library was more actively pursuing technology planning activities. 
Although this was already underway due to various projects, the OnLine at PA 
Libraries project increased the necessity for technology planmng. 

• The library is pursuing new collections development activities to supplement 
Internet use. Directly as a result of Internet access, the library now 
subscribes to a variety of Internet magazines and has purchased numerous 
Internet guides. 

• It has brought in new patrons. The library has attracted new patrons directly 
as a result of the workstation. 

With these benefits, the director considered the project a great success. 



Issues and Recommendations 

While the director expressed appreciation to Bell Atlantic, the Office of 
Commonwealth Libraries, and the SIU, she had the following recommendations: 

• Provide for additional terminals. One terminal, while a good start, is simply not 
enough to meet the needs of both the public and library staff. Library staff 
need dedicated terminals to gain proficiency in the use of the Internet as a 
reference tool, and the public needs additional terminals so as not to endure 
long waiting times. 

• Faster connection. The 28.8 connection is adequate for some searching and 
browsing, but, increasingly, sites are enhancing their graphics and other 
features that require greater bandwidth. 

• Provide more firequent and targeted training. The director acknowledged that it 
is difficult to provide all the system member libraries with appropriate training 
due to the varied technical experience of the library personnel. However, the 
director recommended two types of training: 1) Technical, which would address 
general and advanced computing issues -- e.g., computer configurations, 
printers; and 2) Internet, which would address searching, navigation, and 
reference issues -- e.g., collection development, searching techniques, etc. 



• Electronic collection development. While the director viewed the Internet and 
its vast resources as a welcome addition to the library’s resources, she did not 
consider the OnLine PA workstation and the access to the Internet it provided 
as a substitute for library collection development activities. There is 
inadequate access, time, and knowledge on the part of librarians at present to 
forgo existing printed resources. 
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All the recommendations/issues raised by the director indicate the positive impact 
that the OnLine PA workstation and access to the Internet have provided the library 
— it is simply a case of learning more, getting more access, and having the 
opportunity to integrate Internet-based resources into library activities. 



Library Staff Meeting 

This meeting was attended by three library staff members — a children’s librarian, 
a reference librarian, and a technical services librarian. The three had an average of 
6 years experience in their current positions. Another staff member who had 
substantial OnLine PA experience, including attendance at the spring training 
session, was on vacation during the site visit. 

All stated that they were pleased with the addition of the Internet to the library’s 
resources. Although the library had an Internet connection prior to the OnLine PA 
project, they found the new computer and connection to be a substantial 
improvement over the previous configuration/equipment. 



Types of Users and Uses 

All agreed that it was difficult to classify the “typical” user of the OnLine at PA 
Libraries project. Users were of all types - elementary/high school students, working 
professionals, and retired. The only “missing” users were college students, but the 
librarians attributed that to the better Internet connectivity, access, and equipment 
that the college provided its students. 

The reference librarian has kept some monthly workstation use statistics 
demographics. For the months of February- April 1997, an average of 193 users per 
month used the workstation. Of those, a vast majority, 144 (75%), were 16 or older. 
Interestingly, an average of 62 (32%) users per month were new users, while the rest 
were repeat users. Thus, the workstation steadily attracts new users while 
simultaneously maintaining a solid repeat chentele. 

The types of information sought by users of the workstation reflected the diversity 
of the user community itself. Overall, however, the librarians found users to generally 
search for information on the following topics: 

• Job and employment opportunities; 

• Financial/investment; 

• Travel/vacation; 

• Medical and health; 

• Research for various school assignments; 
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• Individual research; and 

• Government information. 

The range and scope of the researched topic areas reflected both typical and 
expanded uses of other library resources. 



Internet as a Library Resource 

Due to training, access, and familiarity issues (discussed below), the librarians did 
not use the workstation for routine reference questions that could be answered 
through existing library reference material. Rather, the librarians used the Internet 
to get answers to hard-to-find/obscure questions. Examples include: 

• Why are school buses yellow? 

• Why are there those signs with 3 crosses on the highways? 

• What was the content of the Vincent Foster (author’s note: President Clinton’s 
council that committed suicide) suicide note? 

The librarians would not have been able to answer such questions for patrons without 
the Internet. 

The librarians had not engaged in any formal Internet-based resource/collection 
building activities. At most, they create and organize selected bookmarks for 
particular information. One noteworthy, but irregular, activity that the librarians 
engage in is tracking reference questions (the question, and URL used to answer the 
question). Due to the lack of time, however, not all requests are logged or shared with 
other librarians. 



Training. Use, and Frustration 

The librarians lamented the training situation for use of both the computer and 
the Internet. One of the librarians attended the spring training session and found it to 
be “excellent.” The 3 complaints raised about the training session regarded 1) timing - 
- it was too long after the workstation was up and running; 2) distance — the training 
sessions offered were located at a “great distance” from the library; and 3) level of 
instruction — although the librarian from this library did not have difficulty in the 
session, it was her impression that others did due to the level of Internet familiarity 
that was expected from the attendees. 

The librarians were frustrated by the Internet situation within the library. While 
all found the Internet and the OnLine at PA Libraries project to be of great value to 
the library, none could really sit down and learn either well. The library is open 62 
hours per week, and the workstation is in use for nearly all 62 hours. Librarians, 
therefore, can rarely, if at all, practice what they have learned or read about. As one 
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librarian commented, “the best way to learn the Internet is by doing, and we never 
really get a chance to do that because it’s always being used.” 

The lack of familiarity with the Internet and computer made the librarians often 
“feel like idiots” because patrons generally knew more about both than did the 
librarians. This was an uncomfortable turning of the tables for the librarians who 
were used to being the relied-upon experts. 

For the librarians, there is a need for two types of training: 1) General computing 
skills — during which librarians would be instructed in computer use, hardware, etc.; 
and 2) Internet searching and use skills — during which librarians would gain much 
needed “Internet as a tool/resource” instruction. The computing skills training was 
particularly important as users were constantly finding ways to circumvent the 
Fortress security program and change the workstation’s settings. 

In terms of patron training, users currently receive “quick and dirty” introductory 
sessions that consist of basic use skills — mouse movement, use of Netscape, and 
basic searching. The sessions typically last 5-10 minutes. 

Interestingly, however, the librarian who could not attend the meeting had 
organized a variety of user training sessions that were conducted by outside 
corporate sources on a volunteer basis. The training sessions ran monthly for a 
period of 2 years and covered both introductory material as well as coi^ses on specific 
material (e.g., medical information). The courses were offered in the library, with the 
trainers demonstrating course content through a single terminal with an overhead 
display. These sessions no longer run. 



Marketing and Expectations 

The librarians expressed concern over the use of the terminal and marketing. 
Basically, the library could not sustain any more users than the current level. 
Indeed, as mentioned earlier, the library was in the process of adopting a half-hour 
use policy rather than the current one-hour use policy. 

Another draw-back to the publicity - both about the Internet in general and the 
library’s connection in particular - was that many users came to the library with 
false expectations of the scope, nature, and reliability of information available 
through the Internet. Librarians found themselves having to educate users, as best 
as possible, about the Internet itself and the availability of Internet-based 
information. Simply put, users expected the Internet to provide them with 
everything they wanted to laiow about a topic instantly. 



Work Load 



The librarians viewed the OnLine at PA Libraries project as an increase in their 
work load. The librarians provided patrons with Internet instruction, tips, and 
assistance while simultaneously performing their reference desk duties. There was 
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no release time from other activities to pursue Internet instruction, etc. As a result, 
the librarians viewed the workstation as a work generator. 



Benefits 

The librarians pointed to several tangible benefits derived though the OnLine at 
PA Libraries project. These include; 

• Access to a wide source of information that the library could never afford to 
purchase or maintain in its collection; 

• Provision of answers to questions “without having to go outside the library;” 
and 

• Enhancement of the library’s image as an information source in the networked 
environment. 

The librarians considered these benefits to be tremendously important and indicators 
of the project’s overall success. 



Issues and Recommendations 

The staff essentially had two recommendations: 

• Provide librarians with their own workstations. At present, librarians were in 
competition with users for the workstation and the patron was given 
preference. Librarians, therefore, had little chance to hone their computing 
and Internet skills. 

• Provide more frequent and targeted training for staff. The librarians felt that 
they needed regular training sessions that would increase their computing and 
Internet skills to “keep pace with users.” 

Access to more technology and training to gain a better understanding of those 
technologies would serve to “keep the library a vital resource” in the community. 



Luncheon Focus Group with Neighbor Librarians 

The lunch included librarians from three federated system libraries in addition to 
the visited hbrary. One was a library director, one an assistant director, and the third 
was a reference librarian. Each had been in her current position between 4 and 12 
years. They represented libraries had budgets that ranged between $125,000 and 
$500,000, the population served for the libraries ranged between 9,000 and 67,000, 
and the circulation ranged between 85,000 and 450,000. 
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For two libraries, the OnLine PA workstation was the first library access to the 
Internet. One had received the same Internet workstation/connection from the local 
newspaper as had the case site library. All considered the OnLine PA project to be 
successfiil, a “good thing” for their library, and a tremendous resource. 

Users and Uses 



Although the libraries were members of the federated system within the same 
county, they served very different populations: rural, urban, and suburban. As such, 
the libraries served a diverse set of patrons, and the use of the OnLine PA 
workstation reflected that diversity. 

The group estimated that 25-33% of the workstation users were new to the 
library, with the rest being regular library patrons who found the addition of the 
workstation to be of great value. The users ranged from white collar professionals 
who had access to the Internet elsewhere to blue collar workers for whom the library 
was the only source of access to the Internet to students who used the Internet to 
complete school assignments. 

The workstations were in constant use in all the libraries. Two had instituted sign- 
up policies that limited use to an hour at a time. As part of a federated system policy, 
however, all were moving to a half-hour use policy. 

Typical types of information for which users search includes: 

• Job/Emplo 5 rment information; 

• Research activities, both work- and hobby-related information; 

• School assignment-related information for various report and research 
projects; 

• Government information; and 

• Consumer information, such as travel/vacation and automobiles. 

The librarians noted, however, that the range of topics continues to increase as users 
gain in their imderstanding of the Internet as an information source. 



Use Policies 



As mentioned above, the libraries, through the federated system, are adopting 
time limit policies of a half hour. In addition, the libraries, once again through the 
federated system, are in the process of modil^ng the workstation’s acceptable use 
policy. 

The new policy is essentially an attempt to limit the libraries’ 
responsibility/liability when users, particularly minors, access inappropriate 
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material. The policy shifts the responsibility from the library to the parents. The 
librarians expressed concern over using filtering software (as does the case site 
library) that broadly limits access to Internet-based information and did not want the 
role of determining who could access what material on the Internet. 



Gaining Additional Support for the Library 

As mentioned previously, the federated library system, due to the county 
commission’s emphasis on technology/automation, has been successful in gaining 
additional technology-related funding for the systems’s new online public access 
catalog. The participants were optimistic that this support would translate into 
additional support specifically for Internet access. It is unclear, however, as to 
whether this will happen. 



Librarian Access 



All librarians commented that the terminal is “in use all day every day,” clearly 
speaking to the success of and demand for a public library-based public access 
terminal. Because of this success, however, the library staff could not get access 
themselves. As a result, the participants lamented their inability to truly hone their 
Internet skills. This left librarians unable to provide patrons with all the assistance 
that they (both librarians and patrons) wanted to. 



Training 

All the participants had attended the Spring training session and found it to be 
informative and well conducted. They suggested the same two-fold training approach 
that the case site librarians had: 1) Technical training that would address general 
computing use issues including the management of the computer through Fortress; 
and 2) Internet training that would provide “tips and tricks” for searching, browsing, 
and resource location. 

The consultants suggested the creation of a notebook for libraries that could be 
updated at regular intervals. All thought that it would be a great resource for the SIU 
(or Commonwealth Libraries) to create and distribute such a “tips, tricks, and 
resources” notebook to each participating library. The idea of this notebook would be 
to have several sections - e.g., searching, government information - for which 
updates could be distributed on a regular basis (i.e., monthly). This way, librarians 
would have a current resource guide to selected Internet-based information tailored to 
PA hbrary needs. 

Only one of the participating libraries provided regular user training sessions. This 
library provided a weekly half-hour training session for up to 2 users. The session 
provides for basic computing skills such as mouse use as well as basic Internet 
browsing and searching skills. The other libraries provide on-demand 5-10 minute 
introductions to the OnLine PA works tation/Intemet. 



McClure and Bertot 



October 1997 



87 



Evaluation of the OnLine at PA Libraries Proiect 



The 5-10 minute session “satisfies most users.” Librarians commented, however, 
that the “real problem was when we get a total ‘neophyte’ who needs help.” Such 
users could not really be helped with such a brief training session, but “we can’t help 
them more because we’re on reference desk duty or doing something else” in addition 
to assisting with the workstation. 



Benefits 



The librarians cited numerous benefits that their libraries attained due to the 
OnLine PA project that include: 

• Expands collections. Librarians were able to gain access to a wide range of 
information to which they would not have access otherwise. As one librarian 
stated, “it opens a whole new avenue of resources.” Such information included 
daily newspapers, government information, national yellow pages, and state 
and local information. 

• Provides access to the Internet for those who would not otherwise have access. 
Many users of the workstation do not have any other means of access to the 
Internet. As such, the workstation serves as an equalizer to networked 
information. 

• Serves as a catalyst for library and system-wide technology planning 
activities. The influx of new technology — OPAC and the Internet — to the 
library system has created a huge incentive for the libraries to engage in IT 
planning activities. 

• Brings new image/new patrons. The very fact that the libraries have the 
workstation lends them greater credibility within their respective communities. 
Moreover, the added credibility has served to bring in new patrons. 

• Exposes patrons to new technology. Some users were so impressed with the 
Internet that they purchased computers/Internet accounts for use in the 
home. 

The librarians found such benefits to be invaluable to the library and business 
community, and expect that there will be future opportunities afforded the libraries 
because of the project. 



T.d.dne.ci and Recommendations 



Due to the public access nature of the OnLine at PA Libraries project, the lack of 
multiple terminals, and the lack of dedicated Internet time or technical support, the 
hbrarians confironted the following issues: 
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• Lack of access for librarians. Each of the libraries only had the one 
workstation, and this was in constant use by patrons. As a result, librarians 
could not advance their own Internet and computing skills. 

• User expectations. The^ media had “hyped” the Internet and its capabilities as 
a vast information resource to such an extent that librarians were put in the 
position of dealing with false user expectations. 

• Technical support. The lack of dedicated technical support put the librarians 
“on the spot” whenever technical difficulties arose. The librarians 
acknowledged that they were better off than most as they had some 
assistance from the county. The SIU also provided help, but this was not 
always as timely as necessary — having dedicated technical staff would be a 
true plus. 

• Work load. Rather than the Internet alleviating librarian duties, it served to 
increase them. Access to the Internet required that librarians be technically 
literate, network literate, and have vast Internet searching skills. None of this 
made life easier for the librarians; rather, many felt apprehensive as patrons 
were able to navigate the Internet more readily than they could. 

These issues led to the following recommendations: 

• Provide additional terminals for librarians. With the public constantly using 
the workstation, librarians do not get the opportunity to familiarize themselves 
with either the computer or the Internet. 

• Enhance the connection. 28.8 is fine for some types of use, but not for the 
ever-prevalent graphics and video. 

• Provide more training at closer intervals and locations. All commented on the 
need for continual training sessions on a regular basis. Some also felt that the 
locations of the training sessions needed to be closer to the libraries’ region. 

• Create a state-wide resource guide that could be updated at regular intervals. 
At present, there is very little sharing of information, tips, or resources 
between librarians and library systems involved in the project. A guide created 
and managed by central entity with regular updates would be a valuable 
resource. 

• Provide on-premises tefchnical support. When problems arise, help needs to be 
immediate. This is not the case, and frustrates both the librarians as well as 
patrons. 

• Create on-duty Internet exploration opportunities. The librarians commented 
that it would be extremely helpful to have some working time to hone their 
computing/Internet skills uninterrupted and without competing with patrons 
for access to the workstation. 
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Acting on these recommendations would, according to the participants, enhance the 
quality of the OnLine at PA Libraries project beyond its current “wonderful” level. 
None of the librarians expressed dissatisfaction with the project — quite the contrary. 
Rather, they were looking for ways to augment the project and take it to a next level 
of service. 



Focus Group with Users 

The user focus group consisted of four users of the public access workstation. The 
users ranged in age from 27 to 69, and use the workstation between 2 and 20 times 
per month. The focus group lasted for 90 minutes. One user was retired, one was a 
kindergarten teacher, one was unemployed, and one was a social services employee. 



General Impressions 

All participants hailed the arrival of the OnLine PA workstation and access to the 
Internet in the library. To these individuals, the workstation brought “the world to 
our community,” as one participant stated. With such access to essentially a 
“limitless amount of information,” the workstation became a valuable tool to these 
users. This was particularly the case for these users, as none had or could afford a 
computer or Internet account of their own, or could use an Internet-accessing 
computer elsewhere. 

These users also noted that the workstation and access to the Internet increased 
their image of the library as a provider of information and information-related 
services. One user commented that he was “surprised to find the Internet at the 
library,” and that it “keeps me coming back.” 



Types of Information Sought 

All four of the users made different use of the workstation. These uses included: 

• Job search information; 

• Science information for classroom instruction (e.g.. Mars landing information); 

• Legal research; 

• Technology grant information; 

• Medical information; and 

• Vacation planning. 

Each user considered the vast amount of information and resources available to 
them via the Internet (workstation) to be “incredible.” One user summed up the 
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sentiments of the participants when she said “I could never, either through other 
soxirces or through the library, have access to all the things I can now find on the 
Internet.” 



Uses of the Information 



What was perhaps most interesting to the consultant was that each of the users 
acted upon the information he/she sought. For exgunple: 

• The kindergarten teacher actively sought information about the Mars landing, 
printed a number of images, and incorporated the information into lesson plains 
for her students. The students “love it” and thought that it was “cool,” 
particularly the photographs. 

• The social services employee used the workstation for multiple purposes such 
as legal and medical research and technology grant information. This 
individual did volunteer work in a local non-profit organization and used the 
grant information to successfully apply for a computer for that organization. 

• The retired individual planned an entire Hawaiian vacation through the 
Internet one year in advance. He even used the information found about 
airfares and accommodations to negotiate with local travel agents. 

• The unemployed user was actively searching newspapers nationwide for 
emplojonent. He applied for jobs based on his findings. 

As one participant stated, “there is no way that I could have gotten all this 
information without using the Internet.” 



Frustration - Learning and Trairiing 

None of the users was “particularly skilled in computers or the Internet,” as one 
user stated. Some had experience with computers through work or school, but this 
was mainly for such business applications as word processing and spreadsheets. 
Moreover, none had experience with Windows 95 (the operating system on the 
computer). 

The participants noted that very little training was available, if any, from the 
librarians. All commented that the librarians “tried to help,” but that “their help was 
a little here and a little there.” All agreed with what the librarians had said in an 
earlier session - that “they [the librarians] were helping you while they were doing 
other things.” As a result, the librarians could not provide the needed assistance. 

With the training/assistance situation such as it was, the users sought help 
elsewhere - either through Internet guide books that the library purchased, or, more 
often, from the “kids” who were around. All four participants commented that there 
was a group of “kids” (14-17 years old) who were quite good at using the workstation 
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as well as the Internet. Indeed, these “kids” provided the users with a variety of 
assistance such as using the computer, adding bookmarks, searching the Web, 
downloading information, and printing files. In fact, one user commented that the 
“kids” knew how to get around the Fortress security program as well as the filtering 
software. Some even knew how to send e-mail -- a feature that the librarians thought 
they had turned off. 

The lack of training and computer/Internet skills frustrated the users 
tremendously, particularly given the time limit on the workstation. As one user put 
it, “it takes me 35 minutes just to find what I want - every [search engine] is 
different, and I don’t know how to use them all. By the time I find things, my time is 
up.” 



It is interesting to note that none of the users blamed the librarians per se for the 
librarians’ inability to help. The users understood that the librarians were performing 
other duties while also assisting with the workstation. The users, however, 
commented that regular tutorials would be of great assistance. 



Recommendations 

The users, while positive and appreciative of the access to the Internet that the 
workstation provided, had the following recommendations to enhance the library’ s 
Internet services; 

• Add more workstations. The users wanted to see more workstations in the 
library, particularly due to the imposed time limits on the use of the 
workstation. 

• Increase the speed of the connection. Users found the connection to be 
relatively slow, particularly for accessing and printing images. 

• Provide regular tutorials/training sessions. The users commented that the 
tutorials/training sessions do not necessarily have to be conducted by 
librarians - just that there should be training available in the library for the 
connection there. The training should emphasize searching and retrieving 
information. 

• Provide training for the librarians. The users thought that it would be better 
for both the librarians and the patrons if the librarians were more 
knowledgeable in using the Internet and the workstation. Although all 
commented that the librarians tried to be as helpful as possible, they relied 
more on the assistance from “kids” who were in the know about the Internet 
than the librarians. 

Such improvements would enhance an “already wonderful resource” in the library, 
and would keep the users coming back. 
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DOCUMENTING THE SUCCESSES AND IMPACTS 

This chapter documents the successes and impacts that resulted from the OnLine 
at PA Libraries project as described by participants at four different public library 
sites around the state. The information presented in this chapter summarizes 
responses from a range of participants and thus, reports what they said and thought. 
Remarkably, there is much consistency in identif^ng those successes and impacts 
as well as identifying key issues yet to be addressed. Indeed, many of the “successes” 
reported in this chapter are especially powerful as they come directly from project 
participants. Moreover, the successes identified from the case site visits collaborate 
findings reported in Chapter 2 from the various surveys. The next chapter provides 
an overview of key issues, lessons learned, conclusions, and recommendations 
regarding the project based on the consultants’ assessment of findings reported here 
and in the previous chapter. 
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CONCLUSIONS AND DIRECTIONS 



Previous chapters to this report described a range of findings that document the 
successes and importance of the OnLine at PA Libraries project. Over the course of 
the project, significant advances were made in the provision of public access to the 
Internet via rural public libraries; librarians received training on how to best use and 
deploy the public access workstations; and citizens throughout the state were able — 
some for the first time — to access the Internet and take advantage of a new world of 
information and services. 

The purpose of this chapter is to briefly discuss some key aspects of the project 
and selected key project findings. This discussion points to next directions that 
theOffice of Commonwealth Libraries, the public librarians, and others in the state 
might take to build upon the successful work accomplished in the OnLine at PA 
Libraries project. In addition, the project and project evaluation provide some useful 
lessons that can inform “next steps” as the state continues to move forward in its 
support for public library services. 



SUCCESSES, IMPACTS, AND ISSUES 

The findings presented in Chapters 2 and 3 provide a wealth of information 
regarding the implementation of the OnLine at PA Libraries project. The purpose of 
this section is not to repeat the information described previously. Rather, this section 
highlights successes and issues that are especially important to note. 



Successes and Impacts 

From the viewpoint of project participants — be they library directors, library 
staff, users, or local government officials — the project has clearly been successful. 
The list of the various benefits and impacts that have come to the library, the users, 
and the community overall, is significant and consistent across most libraries. Some 
of the most important indicators of the success of the project are that it: 

• Increased the visibility and credibility of participating libraries; 

• Provided valuable publicity for participating libraries as a public access point 
to state-of-the-art technology and information services; 

• Provided a workstation and additional information technology that otherwise 
could not have been purchased or obtained by participating libraries; 
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• Provided access to a range of networked information resources and services 
that otherwise could never have been obtained by participating libraries; 

• Served as a catalyst for participating libraries and their staff to move into the 
electronic networked environment; 

• Removed some of the isolation of the community by providing a “link” between 
the participating libraries’ communities and the ideas, resources, and 
information in a global setting; 

• Linked networked information to users and resolved a range of their 
information needs that otherwise would not have been resolved; there were 
numerous “success stories” of how public access to the Internet greatly 
assisted users; and 

• Brought new users into participating libraries who otherwise may not have 
used “traditional” library services or resources. 

Other key indicators of the success of the project include the following: 

• 26% of users of the public access workstation were new users, i.e., they had not 
previously visited the library. 

• The public access workstation attracts both expert — 44% — and beginning 
users — 48%. 

• 35% of users of the public access workstations had no other means to access 
the Internet except from participating in the OnLine at PA Libraries project. 
Perhaps equally interesting is that the other 65% do have access to the 
Internet elsewhere (e.g., work, home, school), and still choose to use the public 
access workstation. 

• Users of the workstations (on a scale of 1= Very much so and 5= Not at all) 
rated finding the information they needed on the Internet, the ease of using the 
Internet via the library, and the importance of having access via the library 
between 1.7 and 2.2. Overall, therefore, users find that gaining access to the 
Internet, using the workstation, and the information found on the Internet to 
be of value. 

• In terms of major benefits from the project, 48% of participating library 
directors thought the expanded availability to reference sources was critical; 
and 32% thought the major benefit was attracting new users to the library. 
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During the sample week in May 1997 that the user surveys were distributed, one can 
estimate that some 7,600 patrons used the public access workstation in 
participating project libraries. As this report is written in October 1997, the authors 
would estimate that currently the number of users has significantly increased since 
the May survey — and will likely continue to increase especially with the additional 55 
libraries that have been added to the project since the summer of 1997. 

A number of additional successes and impacts are noted in Chapters 2 and 3. But 
it is important to state clearly and without reservation that based on the data from 
the surveys, interviews, site visits, and other data collection efforts, the findings point 
to an overwhelming agreement as to the success of the project. Indeed, the 
enthusiasm and support for the project from participants at the site visits was 
especially overwhelming. 



Im pact on Users 

A key success of the OnLine at PA Libraries project is the impact the project has 
had on the user community. Access to the Internet provides users with numerous 
benefits that include: 

• Access to a wide variety of otherwise unavailable resources, such as 
employment opportunities, research-related material (particularly for 
students), and historical information; 

• Equalization of information resources by eliminating the “small library with a 
small budget” situation. The workstation immediately provides users from all 
walks of life with, hterally, world-wide information resources; 

• Career opportunities due to the technical and Internet skills that users acquire 
through use of the OnLine PA connection; 

• Business opportunities for small businesses, as owners of small businesses 
research and explore various business ventures — both for traditional and 
Internet-based business activities; and 

• Incorporation of Internet-based information by teachers into classroom 
curriculum development. 

The project has, therefore, had a tremendous impact on users and the communities in 
which they reside. Access to the Internet has created business and job opportunities, 
decreased the “information remoteness” often experienced by rural communities, and 
expanded educational capabilities for both students and teachers. Viewed user-by- 
user, these benefits may seem small. In the aggregate, however, these benefits have 
a tremendous impact on Pennsylvania’s communities. 
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Selected Key Issues 

As with most successful initiatives such as the OnLine at PA Libraries project, 
there is also a need to identify a number of important issues. Once again, there are 
numerous key issues described throughout this report. But, this section highlights, 
those issues that the consultants think are especially important. 

Maintaining the Initiative 

Participating librarians are concerned about how they would maintain the 
initiatives begun by the project. Indeed, 90% of the library directors indicated that, 
while they found the public access workstation extremely valuable, their libraries 
could not afford the workstation on their own. For most, the reality is that no 
additional resources were (or will be) forthcoming to add additional terminals, 
purchase upgrades, or otherwise enhance their Internet-based services. Thus, 
initiatives would have to be supported by redeploying the existing limited budgets 
found in most of the libraries. The evidence suggests that while some additional 
support from fund-raising did occur as a result of the project, the libraries did not 
obtain direct budget increases — nor do directors think it is likely that they will receive 
such increases — in the near term. Strategies will need to be developed to continue 
resource support for project objectives as well as next steps resulting from the 
project. 



Enhancing Librarian Skills 

The library staff typically agreed that they need additional training, need 
additional time for such training to occur, and feel very “inadequate” in providing 
assistance to users — who oftentimes knew more than the librarians in accessing the 
Internet and various Websites. If allowed to continue, this situation may result in 
some increased frustration and anxiety on the librarians’ part and injure the future 
development of public access Internet services from the library. Programs for 
additional training of librarians will be essential if the project is to maintain its 
momentum. These programs can be coordinated among the local, district, and state 
library. 



Public Access versus Staff Use of Workstations 



The project was designed to provide public access to the Internet and not 
necessarily to promote staff use of the Internet in the library per se . For a number of 
the staff at these sites, this preference for public access made it difficult to learn how 
to use the workstation (since the public had “priority” for its use); it minimized the 
degree to which the workstation could be integrated into library services (such as 
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administrative applications and reference and referral services); and it limited staff 
opportunity to experiment in the use of the workstation. Library staff need to use the 
lessons from the OnLine at PA Libraries project to now move into how best to 
integrate Internet-based services into the library overall. 



Developing OnLine PA Project II 

The original project has done an excellent job in accomplishing proposed objectives. 
But the project has been so successful that participants — at least at these sites — 
now have a range of expectations for a next, or follow-on, project. They want to build 
on the success of the current project and anticipate the “next steps” for enhancing 
their access to and use of networked information resources and services. There is an 
important opportunity for the state library to propose a strategic initiative to build 
upon the success of the OnLine at PA Libraries project and continue to enhance 
access to Internet-based resources and services through rural public libraries. 



LESSONS LEARNED 

For many of the participants in the project there are important lessons that have 
been learned as to how best to design, implement, and administer an effort such as 
the OnLine at PA Libraries project. Discussed below are some of the more important 
of these lessons that contributed directly to the success of the effort. 



Role of the SIU 

The success of the OnLine at PA Libraries project is due to many people and 
groups. But the role of the Schuylkill Intermediate Unit (SIU) in organizing and 
managing the project certainly was critical for the project’s success. The SIU had a 
very difficult set of responsibilities and tasks that had to be balanced across 188 
public libraries — many of which had unique contexts and situations into which the 
workstations would be operated. 

The use of written agreements between an individual library and the SIU detailing 
responsibilities for involvement in the project was an important component that 
helped everyone to understand who was to do what and who was responsible for what 
(see Appendix A). Equally important, the agreement specified the importance of the 
evaluation component and made clear the expectation that librarians would 
participate in a range of data collection activities. Clearly, the high response rates to 
the evaluation’s various surveys resulted, in part, because of this technique. 
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Results from the evaluation of the training sessions at both the first introductory 
training session and the second session on reference indicate a high quality of 
presentation and content (see Chapter 3). Moreover, it should be noted that 
mounting such training efforts are very difficult when participants have such a wide 
range of background and knowledge related to computing and telecommunications. 

The SIU had to combine a range of technical, managerial, and interpersonal skills 
to complete the project successfully. On the one hand, it ordered, configured, 
delivered, and provided support for all the project workstations. On the other hand, its 
staff also had to provide direct interpersonal support in how to use these 
workstations — in some cases to libraries that had never owned a computer. Thus, 
the SIU had to decide on a basic initial configuration of the workstation that would 
provide the most use and services for the most libraries. 

While some project participants criticized the SIU for that configuration, for the 
security built into the workstation, and other concerns, the reality is that the SIU 
successfully balanced the available resources with the needs of the libraries. The 
SIU did, in fact, obtain and configure the workstations, deliver the various 
workstations, and assist library staff in using them, and planned and delivered a 
range of training sessions. The SIU’s management and support of the project was a 
major factor that contributed to the overall success of the effort. 



Raising Expectations 

An important outcome from the project is the degree to which it successfully 
raised the expectations for new types of library services by both the librarians and 
the citizens of the state. In a number of the libraries that participated in the project, 
there would have been no way that new public access computer workstations with 
Internet connectivity could have been provided to users without the support of this 
project. Once these workstations arrived and were put into use, both the librarians 
and the users had “hands-on” evidence of how these workstations could improve the 
quality of library services in that particular community. 

Thus, a key aspect of the project was its ability to demonstrate how the new 
computing technologies and Internet access could make a difference in the quality of 
library services that directly affected individuals. Rather than talk about how 
important such new services were, the project demonstrated those services. In most 
cases, the new network-based services exceeded the expectations of the librarians and 
users. Thus, the project increased awareness of the use of Internet access, how that 
access can improve library services exponentially (especially in very rural areas), and 
raise expectations of the role of librauies in the information age. 
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To some degree, the success of the project also brings additional pressure to the 
state library to continue to meet these new expectations of librarians and users. 
Many of the participating libraries, now that they are aware and expect this level of 
service from their library, ask how they will be able to support these existing services 
and build new networked-based services. Use logs at these libraries continue to grow 
and library users demand more time and more workstations to access the Internet. 
Librarians worry about how they can meet these demands and provide the “next 
generation” of public access Internet support in their libraries. Time will tell if the 
raised expectations for networked-based library services from users and the 
participating librarians will be translated into better funding and support for 
Pennsylvania hbraries. 



Statewide versus Local Support for 
Internet-based Resources and Services 

Most participating librarians were quite surprised with the demand for public 
access to the Internet as a result of the OnLine at PA Libraries project. In fact, as 
suggested in the site visits (Chapter 3), some were overwhelmed with the demand and 
had to stop advertising the new service because the lacked the staff and resources to 
meet this new demand. Worse, as many told the investigators, there was little 
likelihood that the local governments would be able to find new monies to help the 
libraries meet these new demands and rising expectations for services. 

The lesson appears to be that meeting the demand for high quality electronic 
library services is a statewide issue, not a local issue. While local communities have 
significant affection for their libraries, the demand on local resources for basics such 
as police, fire protection, sewage systems, schools, and others is significant and 
increasing. For many of these local communities there simply are too many other 
demands being placed on the government. It is not that the local governments do not 
wish to support these new initiatives at the library, rather the existing demands for 
resources preclude the library as a possible recipient for increased support. 

Discussions at the user focus groups verify this perspective. The support for the 
library and innovative library services is evident -- the wherewithal for responding to 
these demands in many of these r\iral, economically depressed areas, is not. Indeed, a 
piecemeal solution of helping one rural library at a time simply is not feasible. While 
significant results can be obtained from a “one time shot in the arm” as provided 
libraries by the support of Bell Atlantic, library development is a statewide concern 
and requires statewide support, resources, and solutions. 
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New Roles for Librarians and Libraries 

An important lesson for many of the participating librarians v^^as that maintaining 
the status quo in their library (in terms of use of technology and networked-based 
services) \vas not a viable option. One librarian commented to the investigators that 
“this project has been a rude awakening as to the skills I’ll need to be a good librarian 
in the futiu-e.” Another commented that her library, after participating in the project, 
would never again be “that sleepy, isolated library that didn’t need computers.” 

While there was wide agreement among participants that roles for librarians and 
libraries had to change, there was less agreement on what, specifically those roles 
should be, and how they could be supported to make the transition to these new roles. 
Compounding the problem was the concern about being “forced” to drop some of the 
existing roles in the library to “make room” for the new roles related to operating in 
the global networked environment. 

Thus, the project has also operated as a huge continuing education effort. The 
training provided by the SIU is but one component of the education that has taken 
place for many of the participating librarians. The day-to-day operation of the public 
access workstation, managing the software, the self-learning of how to surf the Web, 
the development of skills to teach others how to use the workstation, and the many 
other skills that evolved occurred in the library, every day, without formal classroom 
instruction. This experience provides an important base by which the librarians can 
now discuss possible new roles both for them and for their library. 

One role to which librarians alluded was that of technology strategists. Librarians 
found that the use of emerging technologies such the Internet, combined with other 
technology-based services such as CD-ROMS and OPACs, required libraries to 
consider their technology deployment and services increasingly as a whole. As one 
library director commented, “we need to look at the whole of our services and 
technology and see what we can integrate, how we can plan, and set some directions.” 

Prior to the OnLine at PA Libraries initiative, too few libraries had given much 
serious thought to the impact of technology on library services in general and how to 
“best” plan for the incorporation of technology-based services into general library 
services in particular. Clearly, a new role for librarians will be technology strategists 
and how best to plan for the provision of technology-based, and especially networked- 
based, resources and services to their clientele. 
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Data Collection Methods and Instruments 

On September 26, 1997, the authors met with staff from the Office of 
Commonwealth Libraries to review the use of the data collection methods and 
instruments with an eye toward determining how best they might be improved. This 
training session provided an opportunity for the authors to share their views on 
aspects of the evaluation that worked well and others that might be improved. 

Figure 4-1 summarizes the range of data collection techniques that the authors 
considered for use in the evaluation. All of the qualitative techniques as outlined in 
Figure 4-1 were used. For the quantitative techniques, network traffic measures and 
Web log file analysis were not used, although considered, for this evaluation. These 
two techniques were not used because there were no central servers where statistics 
could be easily obtained; the cost of placing software on the various workstations to 
maintain counts was, at the time, prohibitive; and the sheer logistics of coordinating 
uniform record keeping across participating libraries while they were still learning 
how to use the workstation simply was not feasible. 

Nonetheless, the evaluation profited by the use of a range of data collection 
techniques. Indeed, an important lesson from the evaluation effort is that the 
combination of qualitative techniques (site visits, focus groups, etc.) with quantitative 
techniques (surveys, logs, etc.) provides a powerful approach by which to compare 
and validate findings. Specific lessons related to the data collection methods and 
instruments include the following: 

• Importance of planning and preparation . The success of these data collection 
instruments resulted, in part, from their careful development and pretesting. 
In addition, specific instructions on how to administer the data collection 
instrument (such as the site visits. Appendix G) are essential. 

• Wide involvement and support . The training efforts on how to administer the 
instrument (such as the user survey. Appendix C) at the second general 
training session is a good example of the importance of the support and 
involvement by the State Library, the SIU, and the district librarians to 
encourage participant involvement in the data collection activities. 

• Multi-level distribution and collection network . Related to the Wide 
involvement and support detailed above, is the availability of various support 
and distribution channels throughout Pennsylvania — i.e., the Intermediary 
Units and the District Libraiy Centers — that can assist in the dissemination, 
collection, and various other data collection activities. Within each of these 
entities are individuals and/or consultants with expertise that can greatly 
assist the state in collecting needed data. 
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• Modify Part II of the user survey . Part II of the user survey (see Appendix C) 
asked users to list the various websites they visited during that session. Most 
did not complete this section. A different approach might be to ask them 
simply to list the most interesting or most useful website they visited during 
that particular session. Or, load appropriate software on the workstation that 
can track the sites actually visited rather than using a survey. 

• Participant questionnaires . At each of the focus groups, group interviews, or 
individual interviews, the evaluators asked participants to complete a short 
questionnaire (see Appendices H and I). These surveys provided excellent 
information describing participants and their background and provided a useful 
context for understanding the discussions. These surveys, however, should be 
made shorter and stress more demographics and background information 
rather than asking participants to assess or list particular issues — which 
typically came out in the discussion. 

• Keyboarding of survey results . The user survey resulted in over 2,000 
responses (see Chapter 3) - all of which needed to be keyboarded for analysis. 
The large response rate was not anticipated by the evaluators. Were it not for 
a contingency plan by the SIU which was able to arrange for the keyboarding 
of all the surveys, only a sample could have been keyboarded and analyzed. 
The lesson is that costs for keyboarding and data analysis of surveys can be 
considerable and require a budget. 

• Participants in the user focus groups . As part of the site visit, the evaluators 
asked the participating librarian to arrange for users of the public access 
workstation to be present at a particular time for a focus group. This 
approach had mixed success in obtaining an adequate number of participants. 
A more effective approach, as it turned out, was that library staff simply 
“snagged” workstation users who were present in the library at the time of the 
focus group and asked them to participate — all of whom, when asked, did 
participate. 

All of these lessons identify strategies that can make future data collection activities 
more effective. 

There is a clear need to replicate the user survey during the fall of 1997. The user 
surveys originally were administered in May 1997. For some libraries they had only 
been online with their workstation for 5 or less months. At the time, use of the 
workstations was increasing steadily. A more accurate estimate of the number of 
users of the workstations would be revealed by administering the survey again, in Fall 
1997 and then once every year. 
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Figure 4-1. Methodologies for On-going Assessment of the OnLine PA 

Project. 





Technique 


Function/Purpose 


Case Sites 


In-depth exploration of selected communities and target audiences in 
those communities, use of and involvement with the project. 

Use findings to inform broader quantitative data collection activities 
such as mail and electronic surveys. 


Content 

Analysis 


Gather various documentation and reports to review historical 
development and evolution of project-related activities. 


Focus 

Groups 


Explore identified key issue areas of project content, services, 
management, and performance. 

Use findings to inform broader quantitative data collection activities 
such as mail and electronic surveys. 


Small Group 
and 

Individual 

Interviews 


In-depth exploration of project content, services, management and 
performance with key project administrators and users. 

Assess the relationship between components of the project and future 
educational use and development of project resources. 

Use findings to inform broader quantitative data collection activities 
such as mail and electronic surveys. 


Critical Path 
Analysis 


In-depth exploration of user-based interactions with project-related 
components, e.g., training, workstation use, and searching. 

Use findings to uncover specific instance issues. Particularly 
appropriate for in-depth analysis of training and use issues. 




Technique 


Function/Purpose 


Mail/Elec- 

tronic 

Surveys 


Further explore identified key issue areas of project content, services, 
management, and performance with broader project population. 

Test findings from qualitative data collection activities with broader 
project population. 


Network 

Traffic 

Measures 


Collect network/terminal traffic use statistics such as users, user 
access points, information and service content use, and network 
server and router load. 

Provides sense of network load, capacity, and what services are used 
with what frequency. 


Web Log File 
Analysis 


Measure Web-based services by the analysis of Web server log files. 

Provides sense of users and locations from which access the services, 
server traffic, type of technology users have, and errors made. 
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In addition, some thought should be given to administering the user survey, or 
some version of that survey, statewide to all public libraries and not only to OnLine at 
PA Libraries. This effort could be contracted out by the Office of Commonwealth 
Libraries or it might become part of the annual survey data solicited by the library 
for Federal and state requirements. Regardless of the approach, the Office of 
Commonwealth Libraries will want to consider very carefully the types of Internet- 
related and electronic services data that should be collected as part of its statistics 
program on a regular basis. 

But the lesson to be learned in the area of data collection techniques is that 
methods can be developed that will identify, describe, and measure the impact of a 
project such as this on libraries, librarians, the local community, and users. Equally 
important is the lesson that additional, ongoing, data collection will be needed to 
monitor the impact of the project, to develop longitudinal data to describe trends 
(such as usage of the workstations), and to continue to demonstrate the importance 
and impact of this project. 



NEED FOR STRATEGIC PLANNING 

This has been an exciting time for many of the libraries participating in the project 
and especially for those visited by the consultants. For a number of these staff and 
communities, the library has been thrust into a very visible and central role in the 
provision of networked information resources and services. But there are conflicting 
demands that will have to be resolved in the very near-term, including the: 

• Need for additional workstations and better connectivity with few additional 
resources to pay for these enhancements; 

• Need to integrate existing and new technologies to provide technology-based 
services as comprehensively and efficiently as possible; 

• Demand for more sophisticated reference and referral services from users and 
inability of library staff to provide such sophisticated assistance; 

• Rethinking and integrating a range of “traditional” library services and 
activities (such as collection development) in the context of networked 
information resources and services; and 

• Considering new organizational approaches for resource sharing among the 
libraries, the district libraries, the state library and other organizations. 

Many librarians noted that their involvement in this project resulted in more, not less 
work. There is some recognition that they had to change in a number of ways to 
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support the success of the project. But there was some concern about how best to 
evolve into this new “networked environment” and how best to obtain resources to 
support the transition. 

The library staff at the site visits are now seeing significant new demands being 
placed on the library, they see the range of opportunities that are possible in 
participating in this networked environment, but the path of how to move from their 
current situation to new roles and services in the networked library setting is unclear. 
They are besieged with competing priorities demanding their attention and the reality 
is that the Ubraries cannot be all things for all people all the time. 

The addition of providing public access Internet services and the recognition of the 
possible networked services and resources that could be provided by the libraries has, 
and could result in a significant increase in the library staffs workload. If libraries in 
the OnLine at PA Libraries project are going to thrive and be successful in this 
evolving networked environment careful planning will be required in setting priorities 
of what activities are most important to be done, and how best those activities can be 
accomplished. 

Thus, there is a need for a statewide rethinking of rxiral public library roles in the 
networked information environment. A strategic initiative should be developed to 
better utilize and leverage resources among the State Library, the district library 
centers, and individual libraries. This strategic initiative will require a vision of the 
future, a plan, a budget, a commitment among the library community, and political 
support at both the state and local level. 



CREATING A NEW FUTURE 

Public libraries in Pennsylvania have an important opportunity to move forward 
in the provision of both traditional and innovative and electronic information services 
to their communities. This opportunity exists because of a number of factors that 
have come into play as this report is written: 

• The statewide attention that has been drawn to the poor support and funding 
of Pennsylvania public libraries by the investigative paper appearing in the 
Philadelphia Inquirer, “Libraries in Distress,” June 1-June 4, 1997. 

• The success of the OnLine at PA Libraries project and the evidence that 
supports that success as described in this report. 

• The current state legislature interest as well as interest from the governor’s 
office in public libraries and the increased awareness that the state 
government may need to take action to support public libraries in more, better, 
and possibly different ways than in the past. 
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But to take advantage of these factors, the public library community, with the 
leadership of the state library, will need to offer a new vision of the role of public 
libraries as a means for improving the quality of life of the state’s citizens, as a 
catalyst for promoting economic development, and as a link to the rest of the world 
for life-long education and learning. This challenge is a key part of the “next project” 
that builds upon the success of the OnLine at PA Libraries project. 
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APPENDIX A 

Application to Participate in OnLine at PA Libraries Project 




Celebrating 250 Years of Service 



Commonwealth Libraries 

PA Department of Education • P.O. Box 1601 • Harrisburg, PA 17105-1601 



May 1996 

INFORMATION MEMORANDUM #96-8 

SUBJECT; Applications to Participate in “OnLine at PA Libraries” 

TO: Directors, Pennsylvania Public Libraries 

\ 

FROM: Gary D. WoL^) . 

Deputy Secrrary of Education 



I am pleased to announce Bell Atlantic has awarded a $750,000 grant to the Pennsylvania Department of 
Education Commonwealth Libraries for public access to the Internet at local public libraries. The name of 
the project is OnLine at PA Libraries and its objective is to provide timely information available on the 
Internet to library users, particularly in rural areas where access to online information is less likely to be 
available. 

Commonwealth Libraries will award equipment grants to libraries wishing to participate in the project. Grant 
fimds will also be used to establish network nodes and alternative communications pathways to ensure the 
access is affordable for small libraries. Schuylkill Intermediate Unit will provide technical assistance and 
consultation throughout the grant and will conduct training sessions for library staff. 

An information sheet and an application for an equipment grant follow this memorandum. Census estimates 
for 1994, used to identify rural counties and municipalities, will be deposited at district library centers and 
made available online on Commonwealth Libraries’ bulletin board. To access the bulletin board: Telnet to 
HSLC.ORG, log on as PABBS, choose CBBS from the menu, follow the instructions on the screen and then 
select PA-LIM. 

Attachments 

GDW:ds 




122 



OnLine at PA Libraries 

a partnership of the Pennsylvania Department of Education 



and Bell Atlantic 



Commonwealth Libraries 

P.O. Box 1601 
Harrisburg, PA 17105-1601 

Application for Public Libraries - Due June 15, 1996 



Library Name. 

Address 

r-ity 



County. 



PA Zip+4 

District Library Center. 



■ j tn Jhp Intsmst throuQh public libraries. This project has 

Commonwealth Libraries. 



1. Does your 
Circle cr.s: 



public library provide a public microcomputer with access to the Internet? 



Yes 



No 



If yes, answer questions 2 and 3. If no, skip to question 4. 

2. What bast describas the public access to the Internet that your library provides? 



Circle all that apply: 
text-based interface only 
dial-up access 



graphical interface (pictures and text) 
SLIP connection 



PPP connection 



3 , Please describe the equipment used to provide publio access to the Internet: 



Circle all that apply: 

PC DOS based microcomputer, 286 or less 
PC DOS based microcomputer, 386 or 486 
PC/Windows microcomputer, 386 or 486 



Apple Macintosh or other Apple 
Modem, 28.8 speed 
Modem, specify speed 



4. 



Printer, specify. 



PC/Windows microcomputer, Pentium 

toSinta^aoh 

areas to calculate population density of members.) 



Persons per square mile of each municipality in service area. List municipality(ies) and check under 
categories: (Data is available at DLCs and Commonwealth Libraries, Library Development Office.) 



Municipality (ies) 
(list below) 


Very Rural 
1 -50/ 
Square Mi. 


Rural 

50.1 - 100/ 
Square Mi. 


Moderate 
100.1 -2000/ 
Square Mi. 


Urban 

2000.1 - 4000/ 
Square Mi. 


Very Urban 
over 4000/ 
Square Mi. 



















































Attach a page if necessary to list all municipalities in the library’s service area. 



In signing this application, the library agrees to the following conditions: 



a. Any equipment received, such as a microcomputer and modem, will be placed in an accessible public 
service area and made available for members of the public to use when the library is open. The library 
will provide a suitable table and chair(s). 

b. The library will provide one phone line to the equipment. 



c. The library will participate in the project at least two years after the grant year and will pay certain start-up 
and continuing costs, estimated to be approximately the following: 



Price 

‘Phone line installation $125 

‘Telephone bill for local line $30/mo. 

‘Internet provider fee $1 5-30/mo. 

‘Software/hardware maintenance $200 

Training/technical support $100 



‘These fees vary depending on the provider. 



First Year 
$ 125 
360 

180-360 

- 0 - 

- 0 - 

$665 - $845 



Following Years 
$ - 0 - 
360 

180 - 360 
200 
100 

$965 -$1195 



d. The library agrees to have at least one public service staff member attend all required training sessions 
sponsored by Commonwealth Libraries in connection with this grant. It is estimated the staff member will 
be required to attend two workshops during the grant year held in locations throughout the state. 



e. The library agrees to participate in public awareness activities and to promote the project to the public 
and the local media. 



I certify the 



agrees to conditions a - e, above, and that the 



(name of library) 

answers to questions 1-4 are correct to the best of my knowledge. 



Signature, Library Director 



Signature, Library Board President 



Name (please print or type) 



Name (please print or type) 



OnLine at PA Libraries 

a partnership of the Pennsylvania Department of Education and Bell Atlantic 



1. What is “OnLine at PA Libraries”? 

Project goals are: 

• To provide public access to the Internet through Pennsylvania’s public libraries 

• To provide library users with timely information for educational, research, economic 
development and recreational purposes 

• To ensure that trained library staff are available to help the public find the information they 
need in an online environment 

This project is made possible ±>y a grant from Bell Atlantic to the Pennsylvania Department of 
Education, Commonwealth Libraries. 

2. What will the project consist of? 

Commonwealth Libraries, part of the Department of Education, will make equipment grants 
to libraries that agree to provide a public access workstation. A typical equipment grant will 
include a multimedia Pentium microcomputer, high speed modem and software. These 
grants will enable a graphical, multimedia interface with the Internet. 

Where libraries already provide public access. Commonwealth Libraries may award grants to 
upgrade that access to a graphical interface to the Internet. 

Commonwealth Libraries will work to establish additional network nodes as appropriate and 
will work to find alternative telecommunications pathways. The objective is to provide access 
for participating libraries via local phone calls and to avoid long-distance charges. 

Training workshops will be conducted at various locations around the state for library staff. 

An outside consultant will evaluate the project at both the local and statewide levels. 

3. Does this project target any particular type of library? 

Yes. The priority is to improve Internet access for populations in rural areas (see definition 
on application). Grants may also be made to libraries in other areas. 

4. What about large libraries? 

Commonwealth Libraries intends to allocate federal funds— Library Services and Construction 
Act Title ll-to support the goals of this project by setting up local area networks (U\Ns) to 
provide access to the Internet for multiple users in district library centers. 
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Eligibility 



• The applicant must be a public library that received state aid in 1996 and that continues to 
meet requirements for state aid in the future. 

• The library director and board president must both sign the application and agree to the 
following: 

to place equipment in an area dedicated to public use and provide a table and chair; 
to provide one phone line dedicated to the project; 

to continue the service for two years after the grant is awarded and pay certain start-up 
and continuing costs; 

to have at least one public service staff member attend all required training sessions; and 
to participate in project public awareness activities and actively promote the project to the 
public and the local media. 

Criteria for Grant Awards 

• The extent to which the library serves a rural population, as measured by the number of 
persons per square mile (see application form). 

• The extent to which the library needs the equipment to provide Internet service, as shown by 
a lack of available public access workstations or by the documented use of existing 
workstations. 

• The availability of affordable, reliable Internet access within a local phone call of the library. 
(In some areas, it is expected that equipment grants will not be made until Commonwealth 
Libraries establishes a network node or identifies a local provider.) 

• The extent to which the library is willing to release staff members for training, as shown by 
the district library center’s assessment. 

Instructions for Applying 

1. Complete the application form and have it signed by the library director and board president. 

2. Send the original application form and three copies to the district library center. 

3. District library centers should forward the original and three copies with a cover letter. In the 
cover letter the district administrator or the consultant librarian should comment on the 
application, discuss its relationship to the district’s long range plan and assess the likelihood 
that library staff will attend training sessions, based on their attendance at district meetings 
and other workshops. 

4. Grant applications must be received by Commonwealth Libraries by June 15, 1996. 

Send applications to; Commonwealth Libraries 

Attention; Barbara W. Cole 

Address U.S. Mail; P.O. Box 1601, Hanisburg, PA 17105-1601 

Address others; 217 Forum Bldg., Commonwealth Ave. & Walnut St. 

(UPS, FedEx, etc.) Harrisburg, PA 17120 
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Department of Information Systems 




March 13, 1996 



Dear OnLine PA Participant: 

One component of the the OnLine PA project is to evaluate the impact, benefits, 
and issues associated with pubhc access to the Internet through the Bell Atlantic- 
provided workstations. The Office of the Commonwealth Libraries selected Dr. 
Charles R. McClure, Distinguished Professor at the School of Information 
Studies, Syracuse University, and Dr. John Carlo Bertot, Assistant Professor in 
the Department of Information Systems, University of Maryland Baltimore 
Cdimty, to conduct the evaluation study. 

One of the many different evaluation project data collection activities is the 
enclosed survey. To assist in the evaluation process, we are asking that you 
complete and return the survey by April 1 . Your participation is extremely 
important and will greatly assist us in determining the impact and issues 
associated with access to the Internet through the OnLine PA equipment. 

Should you'-have any questions regarding the survey, contact John Bertot at 
(410) 455-3883 'phone, (410) 455-1073 fax, or via e-mail at <bertot@umbc.edu>. 
Thank you for taking the time to complete the survey! 



University of Maryland Baltimore County 
5401 Wilkens Avenue 
Baltimore, MO 21228-5398 



Tel. (410)455-3206 
Fax (410)455-1073 
Voice/TTY (4)0) 455-3233 



SURVEY OF ONLINE PA PARTICIPATING LIBRARIES 
Please return by April 1! 



Instructions: The purpose of this survey is to help us document and understand the issues, uses, 
and impacts related to your participation in the Online PA project. The survey is sponsored by the 
Office of Commonwealth Libraries as part of the Online PA project. Please answer the questions 
below by marking the appropriate selection or filling in answers. Your responses will remain 
confidential and will not be attributed to you or your library. 

To return the questionnaire, please fold and staple it so that the self-addressed and stamped 
portion on the lower back faces out and return in the mail. Thanks in advance for you help. Your 
participation is important . 

1. Name of person responding Title: 

Name of hbrary Internet E-mail address: 

2. Origmal interest in participating in O nT.in e PA was motivated primarily by: (check one) 

() Interest on the part of hbrary staff () Availabhity of computing equipment 

( ) Community interest ( ) Board interest 

( ) Other (please specify): 

3. Which one use below best describes the most important benefit resulting from the Online PA 
pubhc access Internet workstation? (check one) 

( ) Expands avahabhity oi reference resources ( ) Expands scope oi overall coUection 

( ) Attracts new users to the hbrary ( ) Can answer specific questions 

( ) Other (please specify): 

4. How much does the hbrary spend per month for telecommunications charges for the Online PA 
pubhc access Internet workstation (note: only for the pubhc access Internet workstation funded 
by the Beh Atlantic grant)? 

$ per month 

5. Please estimate the average numbers of sessions, i.e., a person sits down uses the workstation 
and leaves the Online PA pubhc access Internet workstation, during a recent typical week 
(include those who use it more than once): 

Estimated Number of Sessions per Week Type of User 

Children and students (through age 18) 

Adults (older than 18) 

Library staff 

6. When a first time user wants to use the Online PA pubhc access Internet workstation and 
asks for help, how much assistance do you usuahy provide, on average (in minutes)? 

min utes 

7. To date, how often have you provided formal training sessions (scheduled and advertised in 
advance) for users to learn how to use the Online PA public access Internet workstation or 
better use the Internet? 

() 0 0 1-5 () 6-10 () 11-15 () 16-20 0 21 or more 

8. To date, how many instances of formal advertising or promotion (stories in local paper, 
presentations to local groups, etc.) has your hbrary made regarding the Online PA pubhc 
access Internet workstation? 

() 0 0 1-5 () 7-10 () 11-15 O 16-20 0 21 or more 

[over] 



9. Our library’s OnLine PA Internet connection has been fully operational since: 



(mm/yr) 



10. Please indicate the degree to which you agree or disagree with each of the following statements: 



Agree 

a. Providing pubhc access to the Internet has greatly increased 

the visibility of the library. 1 2 

b. Library staff have very good knowledge and skills related to using 

the computing equipment provided through the OnLine PA grant. 1 2 

c. Library staff have very go<xl knowledge and skills related to using 

the Internet. 1 2 

d. Overall, we are very impressed with the range, extent, and scope of 

reference material available on the Internet 1 2 

e. We have had considerable diflhculties with users accessing and 

reading pornography via the public access Internet workstation. 1 2 

f. Gener^y, the co mm ents from the users of the public access Internet 

workstation are very positive. 1 2 

g. We have had very few technical difficulties with the equipment 

provided by the grant. 1 2 



Disagree 
3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 



1 1 . An anecdote that you can relate that best summarizes the usefulness and impact of the 
OnLine PA pubhc access Internet workstation is: 



12. What is the one., suggestion you would offer to improve the OnLine PA project? 



13. Given what you know now about the OnLine PA workstations and the usefulness of public 
access to the Internet, would you be w illin g to have paid the full amount of the grant in order to 
have that access (assuming the grants had not been available). 

( ) Yes ( ) Yes, but unable to afford without () No ( ) Not sure 

the BeU Atlantic grant 



FIRST CLASS MAIL 



TO: Dr. John Bertot 

Department of Information Systems 
University of Maryland Baltimore County 
1000 Hilltop Circle 
Baltimore, Maryland 21250 
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ONLINE PA USER SURVEY INSTRUCTIONS 



Thank you for assisting us in the Office of Commonwealth Libraries-sponsored 
OnLine PA project evaluation. Through this survey, we hope to get user feedback 
concerning (1) their general familiarity with the Internet, (2) the utility of selected 
Internet-based resources, and (3) their impressions of the Internet in general and OnLine 
PA project in particular. Your help in administering this survey will be critical to our 
ability to get this much needed data. 

Survey Distribution 

Enclosed in this packet you will find 30 copies of the user survey. We are asking that 
you and/or a library volunteer administer this survey between May 19, 1997 and May 25, 
1997. We expect you to obtain 25 completed surveys during that week. If your library is 
not able to get 25 completed surveys, send however many completed surveys your library 
has by the end of the week to John Bertot at the adress below. 

Survey Completion 

To administer the survey, please: 

• Ask users' AS THEY SIT AT THE ONLINE PA TERMINALS if they would be 
willing to participate in the study (our experience demonstrates that putting the 
surveys in a box next to the terminal will not result in a high survey completion 
rate); 

• Explain the purpose of the survey and the need for user-based assessment of the 
OnLine PA project; 

• Assure them of the confidentiality of their responses; 

• Ask willing participants to 

complete Part I of the survey prior to accessing the Internet, 
complete Part II while connected to the Internet, and 
complete Part III after tYieiv Internet session; 

• Put the date and time on the completed survey. 

At the end of the week, or when you have 25 completed surveys, send the surveys to: 

Dr. John Bertot 

Department of Information Systems 
University of Maryland Baltimore County 
1000 Hilltop Circle 
Baltimore, Maryland 21250 ■ 

(410) 455-3883 phone; (410) 455-1073 fax; <bertot@umbc.edu> email 

This is a very important survey — your assistance is essential. Thank you for all your 
help! 
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ONLINE PA USER SURVEY 



Instructions: We are conducting this survey to learn more about your use of the 

Internet and improve services. Please complete Part I of this survey prior to using the 
Internet connection. Complete Part II as you use the Internet, simply by logging the 
various sites and sources you visit while on-line. Complete Part III after you have 
completed your use of the Internet. Please return the questionnaire to the librarian or 
volunteer when you are done. THANKS. 

PART I: Background Information (complete prior to using the Internet) 

I. Age: 2. Gender: ( ) male ( ) female 

3. Zip code for primary place of residence: 

4. Your highest level of education completed is (please check one): 

( ) elementary ( ) high school ( ) community college ( ) college ( ) graduate school 

5. Are you a new user for this library because of the Internet connection? ( ) yes ( ) no 

6. Please estimate the number of months of experience you have using the Internet: 

months 

7. How many times over the last month have you used this library's Internet connection? 
,'^imes 

8. In terms of your Internet skills, do you consider yourself to be (please check one): 

( ) a beginner ( ) an intermediate user ( ) an expert user 

9. In terms of formal training for Internet use, you have had (please check one): 

( ) no training ( ) 1-3 hours of training ( ) more than 3 hours of training 

10. Do you have access to the Internet at (please check all that apply): 

( ) home ( ) school ( ) work ( ) other ( ) none of these 

II. Your primary purpose of using the Internet today is (please check one): 

( ) browsing ( ) homework ( ) recreation ( ) work-related ( ) other 

12. What type of information are you looking for during this session? 



13. If you are willing to participate in a discussion group about your experiences in 
using the library's Internet connection, which would meet locally for about an 
hour at a later date, please provide us with your name and phone number: 



[over] 



PART II: Activities Log while on the Internet (5 most useful sites only) 
Note: complete while you are on the Internet 



Why was this site Number of 

Name or URL of useful? Minutes at 

Internet Site (please check all that apply) this site? 



1. 


( ) content 
( ) links to 

other sources 


( ) other (please describe) 


min. 


2. 


( ) content 
( ) links to 

other sources 


( ) other (please describe) 


min. 


3. 


( ) content 
( ) links to 

other sources 


( ) other (please describe) 


min. 


4. 


( ) content 
( ) links to 

other sources 


( ) other (please describe) 


min. 


5. 


( ) content 
( ) links to 

other sources 


( ) other (please describe) 


min. 



Part III: Your Assessment of Using the Internet (complete after Internet Use). 
Please circle one number for each question. 

Very Much Not at 

So All 



14. While on the internet, I found the information that I 
came to the library to obtain. 

15. While on the Internet, I found information that was 
very useful and interesting. 

16. The Internet information I located will assist me greatly. 

17. The Internet access provided by the library is 
relatively easy to use. 

18. Having access to the Internet via the library is very 
important to me. 

19. The computer and telecommunications equipment all 
worked very well. 

20. 1 found the speed of the Internet, e.g., loading graphics and 
downloading information, to be quite good. 

21. 1 would tell other members of the community about the 
library's Internet connection. 

22. 1 was able to use the Internet without library staff or 
other assistance. 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



23. The one thing this library could do to improve my use of the Internet would be: 
(please describe below) 



24. Please tell us how you found out about the library’s Internet connection: 



Impacts of Public Library Internet Connectivity on Users: 

Methodology 



One component of the overall data collection process will be to obtain (1) an 
estimate of the number and types of users accessing the Internet via the OnLine 
PA terminals in the participating libraries, (2) a description of that use, i.e., the 
types of resources and services users contacted, (3) an assessment of the 
usefulness of those resources and services contacted, and (4) an assessment of the 
usefulness of the OnLine PA project as a means of access to the Internet. 



Bacl^rovmd 

During a sample one week period in May, all participating libraries were 
asked to assist the investigators to collect data regarding the public s use of the 
Internet. Prior to that sample week period, a packet of information that included 
instructions and data collection forms were distributed to participating libraries 
at the OnLine PA spring training sessions. Participants were also informed of 
the purpose, intent, and importance of the user survey to the OnLine PA 
assessment project during these training sessions. 



Data Collection Instrument 

The study team developed the two-page survey as a means for OnLine PA 
terminal users\to provide data concerning (1) their general familiarity with the 
Internet (2) the utility of selected Internet-based resources, and (3) their 
impressions of the Internet in general and OnLine PA project in particular. 

The initial survey was reviewed by selected OnLine PA project staff. Based on 
comments from that review, the study team revised the survey. The revised 
survey was then pre-tested by two MLS students at the School of Information 
Studies, Syracuse University. Comments and feedback from this pre-test were 
then incorporated into the final survey. 

Survey Distribution Technique 

During the one-week sample period in May, all participating libraries asked 
up to 25 users of their OnLine PA terminal to complete a survey. A librarian or 
volunteer gave the users the survey, explained to them the purpose for collecting 
the data, and instructed the users in the completion of the survey. 

The librarian or volunteer instructed the users to complete Part I of the survey 
prior to accessing the Internet, complete Part II while connected to the Internet, 
and complete Part III after their Internet session. At the end of the sample week 
or having completed 25 surveys (which, ever came first), all completed 
questionnaires were returned to the investigators for analysis. 
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Online at PA Libraries Initial Training Evaluation Survey 



Please take a few minutes of your time to complete this questionnaire. We are interested 
in the degree to which the training you just received met your needs for being able to use 
the Internet when you go back to your library. Thanks very much for taking the time to 
fiU-out this survey. 

1. Participating Library: 

2 . Population of legal service area served by your library: 

3. Are you a: Q Professional librarian (e.g., ALA/ MLS)? Q Other? (Please describe): 

4. How many yecirs have you been a public librarian? Years (Rounded to nearest whole year) 

5. Overall, how would you rate your Internet navigation skills? (Check [X] one only) 

Q Below Average Q Average Q Above Average Q Excellent 

6. Overall, how would you rate your personal computer use skills? (Check [X] one only) 

Q Below Average Q Average Q Above Average Q Excellent 



7. The three (3) most important goals I had for this training were: 
a. 






b. 



ERIC 



□ Didn't have any goals/expectations 



8. Please assess the initial Online at PA Libraries training session along the following (PLEASE CIRCLE ONE 
NUMBER FOR EACH ITEM): 



a) My goals for this training session were met 

b) The ideas presented in this training session were 

very clear 

c) There was sufficient opportunity to ask questions 

during the training session 

d) Overall, the trainer(s) was well prepared 

e) There was too much material covered in this training 

session 

f) I feel like I now have a good basic understanding of how 

to works 

g) Please estimate the degree of difficulty you will have in 

configuring the hardware/software for this project 
when you return to your library 



Not 
At All 

1 

1 

1 

1 

1 

1 

1 



2 

2 



2 

2 

2 

2 



3 

3 



3 

3 

3 

3 



4 

4 



Very 
Much So 

5 
5 



4 

4 

4 

4 



5 

5 

5 

5 



Don't 

Know 

□ 

□ 

□ 

□ 

□ 

□ 

□ 



9. Please identify the extent to which you agree with the following (PLEASE CIRCLE ONE NUMBER FOR 
■ EM): 



EACH ITEI^ 



a) There is a great deal of interest in this project at my 

library 

b) Internet-accessible resources can be useful in a rural 

library context 



Strongly 

Disagree 

1 2 



1 



3 

3 



Strongly 

Agree 

4 5 



Don't 

Know 

□ 

□ 



(over) 
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10 Atthebegianingof the day, the degree to which I understood the following was (PLEASE CIRCLE ONE 

NUMBER FOR EACH ITEM): 



Not At All 



URL Addressing 1 2 

Netscape Navigation 1 2 

Dialing Internet Service 1 2 

Provider 

Setting PC/ Printer 1 2 

Printing Documents 1 2 



A Great Deal 



3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 



11. At the end of the day, the degree to which I understand the following is (PLEASE QRCLE ONE NUMBER FOR 
EACH ITEM): 



Not At All 



A Great Deal 



URL Addressing 1 

Netscape Navigation 1 

Dialing Internet Service 1 

Provider 

Setting Up PC/Printer 1 

Printing Documents 1 



2 

2 

2 

2 

2 



3 

3 

3 

3 

3 



4 

4 

4 

4 

4 



5 

5 

5 

5 

5 



12. The degree to which I can now go back to my library and USE the following is (PLEASE CIRCLE ONE 
NUMBER FOR EACH ITEM): 

Not At All A Great Deal 



URL Addressing 1 

Netscape Navigation 1 

Dialing In ternerService 1 

Provider 

Setting PC/ Printer 1 

Printing Documents 1 



2 

2 

2 

2 

2 



3 

3 

3 

3 

3 



4 

4 

4 

4 

4 



5 

5 

5 

5 

5 



1 3. In the space below, please share any ideas you might have as to how future PA Access training sessions can be 
enhanced to better meet your/your library's needs: 



THANK YOU FOR YOUR PARTICIPATION! 



O 

ERIC 



Questions on completing this survey? Contact 
John Bertot 

Department of InformationSystcms 
Un. of Maryland Baltimore County 
1000 Hilltop Circle 
Baltimore, MD 21250 
(410) 455-3883 phone 
(410) 455-1073 fax 
<bertot<2\imbcedu>e-mai! 
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OnLine at PA Libraries 1997 Spring Training Evaluation Survey 



Please take a few minutes of your time to complete this questionnaire. We are interested 
in the degree to which the training you just received met your needs for being able to use 
the Internet to answer reference questions. Thanks very much for taking the time to 
complete this survey. 

1. Participating Library: 

2. Are you a: Q Professional librarian (e.g., ALA/MLS)? Q Other? (Please describe): 



3, Overall, how would you rate your Internet navigation skills? (Check (X) one only) 
Q Below Average Q Average Q Above Average Q Excellent 



4. Please assess this OnLine at PA Libraries training session along the following (PLEASE CIRCLE ONE 



NUMBER FOR EACH ITEM): 


Strongly 

Agree 








Strongly 

Disagree 


Don't 

Know 


a) 


My goals for this training session were met 


1 


2 


3 


4 


5 


□ 


b) 


The ideas presented in this training session were 
very clear 


1 


2 


3 


4 


5 


□ 


0 


There was sufficient opportunity to ask questions 
during the training session 


1 


2 


3 


4 


5 


□ 


d) 


CKferall, the trainer(s) was well prepared 


1 


2 


3 


4 


5 


□ 


e) 


There was tod'tnuch material covered in this training 
session 


1 


2 


3 


4 


5 


□ 


f) 


I feel like I now have a good basic understanding of how 
to use the Internet as a reference tool 


1 


2 


3 


4 


5 


□ 


g) 


I will now be able to use the Internet to answer reference 
questions 


1 


2 


3 


4 


5 


□ 


h) 


Tlie Internet will be a welcome addition to our library's 
reference tools 


1 


2 


3 


4 


5 


□ 



5. The degree to which I understood the following was (PLEASE CIRCLE ONE NUMBER FOR EACH ITEM): 





At the Beginning of the Day 


At the End of the Day 




Not 
At All 








A Great 
Deal 


Not 
At All 






A Great 
Deal 


a) Searclung the Web 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


b) Using Web-based search engines 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


c) Accessing government documents on 
the Web 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


d) Identifying/using reference resources 
on the Web 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


e) Creating and organizing bookmarks 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


f) Downloading Web files to a diskette 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


gj Downloading software from the Web 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


h) Using Fortress 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 
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6. Now that I have had this training, I will go back to my library and begin offering Internet training to (PLEASE 
CHECK [X] ALL THAT APPLY): 

□ Other library staff □ The public Q Local government officials 

Q Will not offer additional Q Other (please describe): 

training 

7. Overall, I consider the success of the Online PA project to date to be (PLEASE CIRCLE ONE NUMBER): 



Very 






Very 


Successful 






Unsuccessful 


1 


2 


3 


4 5 



THANK YOU FOR YOUR PARTICIPATION! 



Questions on completing this survey? Contact: 
John bertot 

Department of Information ^sterns 
Un. of Maryland Baltimore County 
1000 Hilltop Circle 
Baltimore, MD 21250 
(410) 455-3883 phone 
(410) 455-107i fax 
<bertot®umbc.edu> e-mail 
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HELP DESK LOG 



Date: / / Time: 


AP Lib. Code: 


Library: 


Caller: 


Caller’s Title: 


Phone: ( ) 


Fax: ( ) 


lU 29 Technical Support Specialist: 



Equipm ent List — 

Brand/Model of Computer #1: 

Brand/Model of Computer #2: 

Brand/Model/Number of CD-DRIVES: 

□ Stand-alone □ LAN- Workstation □ Optical Server □ Non-Opcical Server 

□ Standard LANtastic □ Novell Netware ver. □ Other: . 

□ PC/MS DOS 3.3 □ PC/MS DOS 5.xx □ PC/MS DOS 6.xx □ Other: _ 



Problem Description — 



First Contact/Comments — 

























Problem resolved on / / ; .M. 

Problem resolved by: 



Follow-up Call #1 - Date: / / ; Time: — 

























Follow-up Call #2 - Date: / / ; Time: — 












- 














Follow-up Call #3 - Date: / / ; Time: — 
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APPENDIX G 



May 20, 1997 



Ms. 

Participating Library 

Address 

City, PA ZIP 

Dear: 

Thank you for agreeing to participate in our site visits for the OnLine PA project 
evaluation. I enjoyed speaking with you last week about our forthcoming visit to 

on June 5, 1997. The purpose of this letter is to describe the nature of 

the visit in a bit more detail, provide additional information that may help you to 
organize the visit, and to ask for some logistical information to help us get there. 

As I indicated on the phone, there are four main parts to the site visit. Let me 
briefly describe each one so that you will know what to expect. 



1: Meeting with the Director 

We would like to spend the first session with you, say from 9:00 AM- 10:15 AM. 
The objectives of this session are to obtain an administrative overview of your 
library’s role in the project; review costs, benefits, and issues related to your 
involvement; identify specific impacts from involvement on the library, library staff, 
and the local community from involvement; and discuss the library’s next steps in 
both the project and in providing public access to the Internet. 

We are especially interested in your views on the costs and benefits from 
participating in the project. Would you do it again if asked? Has participation really 
affected the community and how? Has the OnLine PA connection increased visibility 
and use of the library? Will you leverage this initial Internet connection and Internet 
services and expand them in the future? 



2: Meeting with Library Staff 

This meeting t 5 rpically would go from 10:30 until 11:45 AM. Please invite ALL 
library staff (and volimteers if appropriate) who have been involved in the OnLine PA 
project at the library. By involved, we mean anyone that has received training, 
anyone who has provided instruction or training to users; or anyone who has operated 
or used the OnLine PA connection. During this session, the topics we would like to 
discuss are: 

• How would you describe the users and t 5 rpes of uses being made of the public 
access Internet connection? 



• What kind of training does the library provide to users? 

• To what degree do you (library staff) use the OnLine PA connection and for 
what do you use it? 

• Please describe some of the most important “successes” and “failures” you 
have seen in using the OnLine PA connection? 

• What are the most important benefits and impacts that have resulted to you, 
the library, and the community from having the OnLine PA connection? 

The meeting should be held in a meeting room or at a conference area in the library 
where we will not disturb anyone else. If we need to meet with them individually in 
orcer to operate services points during this time, such is fine. We hope that you will 
also attend this meeting but will encourage others to talk and offer their views since 
we will have already met with you! 



3: Meeting with Local Librarians 

Please invite up to 7 librarians in neighboring communities to meet with us for 
limch and discussion. The librarians should also be participants in the OnLine PA 
project. The librarians can be directors or staff — as long as they have some 
experience with OnLine PA and provision of the Internet services in their library. 

If appropriate please make reservations for all of us — John and I plus however 
many agree to participate. We should probably all meet directly at the designated 
restaurant rather than at the library to save on time. The closer, quicker, and easier 
the restaurant is to get into and out of, the better! Tell everyone that the lunch is on 
us. We would expect to have limch and return to the library by 1:45. 

The same general topics that we hsted for discussion with the staff will also be the 
topics that we will raise during lunch. So if they want to know the topics in advance, 
please feel free to provide them with the topics listed in item 2 above. It may be 
useful to remind them about the meeting and its location a couple of days prior to the 
time. 



4: Meeting with Local Users 

This meeting is very important as a prime objective of the evaluation is to talk to 
users of the OnLine PA public access connections. If we schedule the meeting for 
2:00 we should finish by 3:30 PM. Again, the meeting should be held in a meeting 
room or at a conference area in the library where we will not disturb anyone else. We 
will ask that the library director introduce the meeting and then leave. 

We would suggest that you start NOW to arrange for users to attend this focus 
group session. We hope that you will be able to identify regular users of the OnLine 
PA Internet connection, that they are knowledgeable about the use of the connection, 
and willing to share their experiences with us. It is fine if they use the Internet from 
their home as long as they are users of the connection at the library too! Users can be 



young or old, students, whatever — as long as they have used the library's OnLine PA 
connection and are willing to discuss their experiences. In fact, users of the O nlin e PA 
project, but from other libraries in the area, are also welcome to participate in the 
session if possible. 

There are a number of strategies to use in organizing this session. One is to have 
an announcement of the meeting, at the computer, asking users to participate in a 
discussion about their use of the On-line PA Internet connection. Indicate that those 
interested should come to the reference desk to sign-up. Then get their name and 
phone number as confirmation. You should probably get about 15 users to agree to 
attend in advance as that usually results in about 9 that actually come. 

In conjunction with the above approach it is usually better to personalize the 
invitation by going to those using the Internet connection (especially users that you 
or the staff know) and asking them directly to participate. Then get their name and 
phone number as confirmation. Two-three days before the meeting, please give them 
all a reminder call letting them know we are looking forward to seeing them at the 
discussion session. Bribe them with the high quality cookies we will bring! 

Please stress to these users the importance of their participation as it is essential 
to document their views on the use and impact of providing this new Internet service. 
Indicate that information fi-om the study will help the library plan for future Internet 
services and possibly help the library and the state obtain additional funding to 
support such services. Typical topics we will discuss with the users are: 

• How did they learn of the OnLine PA Internet co nn ection? 

• How often do they use the connection and what t 5 q)es of information resources 
and services do they use? 

• Do they use it for professional, personal, or recreational reasons — can they 
provide examples of specific uses and how those sources helped (or did not help) 
them? 

• What have been the impacts and benefits of using the library’s OnLine PA 
connection? 

• Did they encounter any difficulties in using the workstation or the software? 

• How might the OnLine PA Internet connection and services be improved? 

• How have they learned to use the OnLine PA Internet connection? 

• What do you recommend the library do to continue to improve this service? 

• Do you think the local community would be willing to support the public access 
Internet connectivity directly? 

This is a general flavor of the t 3 q)es of topics we will explore with them — others may 
come up depending on their knowledge and interest. At 3:30 PM we will end the 
meeting and thank them very much! 
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In Preparation 

Prior to our coining to the library I would greatly appreciate it if you would send 
me information on the following: 

• Although we have some basic statistical information about your library, we 
would appreciate it very much if you would send me a “care package” of any 
information about the library that you think would be useful for us to know. 
This could include a budget (especially any cost items specifically related to the 
project), brochures related to the OnLine PA project, promotional information 
or news clippings about the project, user logs or sign-up sheets, etc. 

• Map with directions on how to find your library and where to park if it is not 
obvious. 

• Your recommendation for a hotel/motel (with phone number) where we might 
stay for the night. 

Please fax me information on hotels/motels and a map at (fax no.). Please send the 
care package to me at (address). 



* * * * 

I think the above about covers the information we need to provide you. If you 
have any questions you can call me at my office (phone no.) where I also have voice 
mail or fax me (phone no.). We really appreciate your help with participating in the 
site visit. THANKS, and we are looking forward to meeting you at 9:00 AM on June 
Xth. 



Sincerely, 



Chuck McClure 



cc: 



John Bertot 
Barbara Cole 
Jim Fogarty 
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Participant Questionnaire 



OnLine PA Evaluation Focus Group/Interview Participant Questionnaire 

Thank you for participating in the On-line PA evaluation. Please take the time to 
complete this supplemental questionnaire before you leave today. The information you 
provide will not in any way be associated with you. 

1. Name: 

2. Institution: 

3. Phone/fax number with area code: ( ) p / ( ) f_ 

4. Internet e-mail address: 

5. Current position/years in position: L years 

6. When did your O nLin e PA computer first become available to the public (e.g., up and 
running with Internet access)? 

(mm/yr) 

Approximately how many hours per week is the OnLine PA computer available 
for public use? 



7. 



8 . 



hours/week 



Please estimate the niimber of people using the OnLine PA computer for Internet 
access per week: 



people/week 

Was the OnLine PA computer the first mvdtimedia computer in your library available 
for public use? 

Yes No 

Overall, how would you rate your Internet navigation skills? 

_ Poor Below Average Average Above Average Excellent 



9. Please list the two most significant benefits of the OnLine PA initiative to your 
community: 

( 1 ) 

( 2 ) 



10. Please identify the two most significant factors. that affect your library’s ability to 
support the OnLine PA initiative: 

( 1 ) 

( 2 ) 

ERIC 
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OnLine PA Evaluation User Focus Group/Interview Participant Questionnaire 

Thank you for participating in the OnLine PA evaluation. Please take the time to 
complete this supplemental questionnaire before you leave today. The information you 
provide will not in any way be associated with you. 

I. Age: 2. Gender: ( ) male ( ) female 

3. Current Occupation: 

4. Your highest level of education completed is (please check one): 

( ) elementary ( ) high school ( ) community college ( ) college ( ) graduate school 

5. Are you a new user for this library because of the Internet connection? ( ) yes ( ) no 

6. Did you first access the Internet through the Ubrary’s OnLine PA Internet cormection? 
□ Yes □ No 

7. How many times over the last month have you used this Ubrarys Internet connection? 
times 

8. In terms of your Internet skills, do you consider yourself to be (please check one): 

( ) a beginner ( ) an intermediate user ( ) an expert user 

9. In terms of formal training for Internet use, you have had (please check one): 

( ) no training ( ) 1-3 hours of training ( ) more than 3 hours of training 

10. Do you have access to the Internet at (please check ali that apply): 

( ) home ( ) school ( ) work ( ) other ( ) none of these 

II. I primarily use the Ubrary’s OnLine PA Internet connection for (please check one): 

( ) browsing ( ) homework ( ) recreation ( ) work-related ( ) other 

12. The two most significant benefits of the OnLine PA initiative are: 

( 1 ) 

( 2 ) 

13. The two most significant improvements to the Ubrary’s OnLine PA Internet connection 
that I would Uke to see are: 




( 1 ) 

(2) 
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